Text to SQL

result = evaluator.evaluate(
    eval_templates="text_to_sql",
    inputs={
        "input": "List the names of all employees who work in the sales department.",
        "output": "SELECT name FROM employees WHERE department = 'sales';"
    },
    model_name="turing_flash"
)

print(result.eval_results[0].output)
print(result.eval_results[0].reason)

import { Evaluator, Templates } from "@future-agi/ai-evaluation";

const evaluator = new Evaluator();

const result = await evaluator.evaluate(
  "text_to_sql",
  {
    input: "List the names of all employees who work in the sales department.",
    output: "SELECT name FROM employees WHERE department = 'sales';"
  },
  {
    modelName: "turing_flash",
  }
);

console.log(result);

​

What to do If you get Undesired Results

• Ensure the SQL syntax is correct and follows standard conventions
• Verify that all tables and columns referenced match the database schema implied by the natural language query
• Check that the query filters for exactly the data requested (no more, no less)
• Make sure appropriate joins are used when multiple tables are involved
• Confirm that the query handles potential edge cases like NULL values appropriately
• Use the correct data types for values in comparisons (e.g., quotation marks for strings)
• For complex queries, consider breaking them down into simpler parts for troubleshooting

​

Comparing Text to SQL with Similar Evals

• Task Completion: While Text to SQL focuses specifically on converting natural language to SQL queries, Task Completion evaluates whether a response completes the requested task more generally.
• Evaluate Function Calling: Text to SQL evaluates SQL generation specifically, whereas Evaluate Function Calling assesses the correctness of function calls and parameters more broadly.
• Is Code: Text to SQL evaluates the correctness of SQL generation, while Is Code detects whether content contains code of any type.