> ## Documentation Index
> Fetch the complete documentation index at: https://futureagi.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# LLM Function Calling

> Evaluates the accuracy and effectiveness of function calls made by LLM. It checks whether the output correctly identifies the need for a tool call and whether it accurately includes the tool with the appropriate parameters extracted from the input.

<CodeGroup>
  ```python Python theme={null}
  result = evaluator.evaluate(
      eval_templates="llm_function_calling",
      inputs={
          "input": "Get the weather for London",
          "output": '{"function": "get_weather", "parameters": {"city": "London", "country": "UK"}}'
      },
      model_name="turing_flash"
  )

  print(result.eval_results[0].output)
  print(result.eval_results[0].reason)
  ```

  ```typescript JS/TS theme={null}
  import { Evaluator, Templates } from "@future-agi/ai-evaluation";

  const evaluator = new Evaluator();

  const result = await evaluator.evaluate(
    "llm_function_calling",
    {
      input: "Get the weather for London",
      output: '{"function": "get_weather", "parameters": {"city": "London", "country": "UK"}}'
    },
    {
      modelName: "turing_flash",
    }
  );

  console.log(result);
  ```
</CodeGroup>

| **Input** |                    |          |                                                                |
| --------- | ------------------ | -------- | -------------------------------------------------------------- |
|           | **Required Input** | **Type** | **Description**                                                |
|           | `input`            | `string` | input provided to the LLM that triggers the function call.     |
|           | `output`           | `string` | LLM's output that has the resulting function call or response. |

| **Output** |            |                                                                                                                                                                      |
| ---------- | ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|            | **Field**  | **Description**                                                                                                                                                      |
|            | **Result** | Returns Passed if the LLM correctly identified that a function/tool call was necessary, or Failed if the LLM did not correctly handle the function call requirement. |

***

### What to Do When Function Calling Evaluation Fails

Examine the output to identify whether the failure was due to missing function call identification or incorrect parameter extraction. If the output did not recognise the need for a function call, review the input to ensure that the function's necessity was clearly communicated. If the parameters were incorrect or incomplete.

Refining the model's output or adjusting the function call handling process can help improve accuracy in future evaluations.

***

### Differentiating Function Calling Eval with [API Call](/future-agi/get-started/evaluation/builtin-evals/api-call) Eval

The API Call evaluation focuses on making network requests to external services and validating the responses, while Evaluate LLM Function Calling examines whether LLMs correctly identify and execute function calls.

API calls are used for external interactions like retrieving data or triggering actions, while function call evaluation ensures that LLMs correctly interpret and execute function calls based on input prompts.

They differ in validation criteria, where API calls are assessed based on response content, status codes, and data integrity, the function call evaluation focuses on the accuracy of function call identification and parameter extraction.