> ## Documentation Index > Fetch the complete documentation index at: https://futureagi.mintlify.site/llms.txt > Use this file to discover all available pages before exploring further. # Create a New Test Run > Creates and configures a new test run, associating it with scenarios, an agent definition, and detailed evaluation configurations. ## OpenAPI ````yaml /openapi.json post /simulate/run-tests/create/ openapi: 3.0.0 info: title: FutureAGI Unified API version: 1.0.0 description: >- Complete API documentation for FutureAGI platform - Simulate module (scenario management, agent definitions, test executions, call analytics) and Model Hub Evaluations API (evaluation templates, playgrounds, metrics, and execution tracking) contact: email: support@futureagi.com servers: - url: https://api.futureagi.com description: Production API security: [] tags: - name: Health description: Health check operations for monitoring server status - name: Authentication description: User authentication and token management - name: Scenarios description: Test scenario management and execution - name: Agent Definitions description: Agent definition CRUD operations - name: Agent Versions description: Agent version control and management - name: Simulator Agents description: Simulator agent operations - name: Run Tests description: Test execution management - name: Test Executions description: Test execution tracking and analytics - name: Call Executions description: Individual call execution details - name: Call Transcripts description: Transcript management and retrieval - name: Personas description: Persona management for testing - name: Analytics description: Analytics and reporting - name: Export Simulate description: Data export operations - name: Datasets description: >- Operations related to datasets, including creation, modification, and data management. - name: Eval Groups description: Evaluation group management - name: Eval Templates description: Base evaluation template operations - name: Custom Eval Templates description: Custom evaluation template CRUD operations - name: Eval Playground description: Test and run evaluations in playground environment - name: Eval Logs & Metrics description: Evaluation logs, metrics, and execution tracking - name: Eval Configuration description: Evaluation configuration and templates retrieval - name: API Keys description: API key management - name: Prompt Workbench description: Prompt template version labeling and lookup by name, version, or label paths: /simulate/run-tests/create/: post: tags: - Run Tests summary: Create a New Test Run description: >- Creates and configures a new test run, associating it with scenarios, an agent definition, and detailed evaluation configurations. operationId: createRunTest requestBody: required: true content: application/json: schema: type: object required: - name - scenarioIds - agentDefinitionId properties: name: type: string description: A unique name for the test run. description: type: string description: An optional description for the test run. scenarioIds: type: array items: type: string format: uuid description: A list of scenario UUIDs to be included in this test run. agentDefinitionId: type: string format: uuid description: The UUID of the agent definition to be tested. agentVersion: type: string format: uuid description: >- The specific UUID of the agent version to be tested. If not provided, the active version will be used. nullable: true evalConfigIds: type: array items: type: string format: uuid description: >- A list of existing evaluation configuration UUIDs to associate with this test run. evaluationsConfig: type: array items: $ref: '#/components/schemas/EvaluationConfig' description: >- A list of new, detailed evaluation configurations to create and associate with this test run. datasetRowIds: type: array items: type: string format: uuid description: A list of specific dataset row UUIDs to test against. enableToolEvaluation: type: boolean description: Flag to enable tool evaluation for this test run. default: false examples: create-run-test-with-evals: summary: Example of creating a new test run with multiple evaluations value: name: new-run-test description: '' scenarioIds: - fae7d086-6466-4b40-b21f-13bb7e1d83fe agentDefinitionId: 87a193df-12a6-46e1-860d-d18ddb4a00cf agentVersion: 117efec9-5e9b-4e9e-9272-cf171b6e4af1 evalConfigIds: [] evaluationsConfig: - name: task_completion templateId: 5419b2e4-f155-4f0f-846f-0a3f848a74be templateName: task_completion mapping: input: transcript output: transcript config: mapping: input: transcript output: transcript config: {} reasonColumn: true description: >- Measures whether the model fulfilled the user's request accurately and completely. type: futureagi_built requiredKeys: - input - output tags: - TEXT - FUTURE_EVALS - AUDIO errorLocalizer: true model: turing_small eval_group: 10a3037b-5893-4997-a5d5-9d058aae10d1 - name: is_polite templateId: 122a4e83-4c5e-4a17-bcfc-1d29affba6f9 templateName: is_polite mapping: output: transcript config: mapping: output: transcript config: {} reasonColumn: true description: >- Ensures that the output maintains a respectful, kind, and non-aggressive tone. type: futureagi_built requiredKeys: - output tags: - TEXT - FUTURE_EVALS - AUDIO errorLocalizer: true model: turing_small eval_group: 10a3037b-5893-4997-a5d5-9d058aae10d1 datasetRowIds: [] enableToolEvaluation: true responses: '201': description: The test run was created successfully. content: application/json: schema: $ref: '#/components/schemas/RunTest' '400': description: Bad Request. The request payload is invalid. content: application/json: schema: $ref: '#/components/schemas/Error' '401': description: Unauthorized. Authentication credentials were not provided. '404': description: >- Not Found. The user's organization or other specified resources could not be determined. content: application/json: schema: $ref: '#/components/schemas/Error' '500': description: Internal Server Error. An unexpected error occurred. content: application/json: schema: $ref: '#/components/schemas/Error' security: - ApiKeyAuth: [] SecretKeyAuth: [] components: schemas: EvaluationConfig: type: object required: - name - templateId - mapping properties: name: type: string description: A user-defined name for this evaluation instance. templateId: type: string format: uuid description: The UUID of the base evaluation template. templateName: type: string description: The name of the base evaluation template. mapping: type: object additionalProperties: type: string description: >- Maps the required keys of the evaluation to the available data columns (e.g., 'transcript'). config: type: object properties: mapping: type: object additionalProperties: type: string description: Redundant mapping object inside the main config. config: type: object description: Additional nested configuration, often empty. reasonColumn: type: boolean description: >- If true, a column for the evaluation reasoning should be included. description: type: string description: Description of the evaluation's purpose. type: type: string description: The type of the evaluation. enum: - futureagi_built - user_built - custom requiredKeys: type: array items: type: string description: A list of input keys that the evaluation requires. tags: type: array items: type: string enum: - TEXT - AUDIO - SAFETY - RAG - HALLUCINATION - FUNCTION - LLMS - CUSTOM - FUTURE_EVALS description: Tags for categorizing the evaluation. errorLocalizer: type: boolean description: Flag to enable error localization. model: type: string description: The model used to perform the evaluation. enum: - turing_small - turing_large - gpt-4 - gpt-3.5-turbo - claude-3-opus eval_group: type: string format: uuid description: The UUID of the evaluation group this configuration belongs to. nullable: true RunTest: type: object properties: id: type: string format: uuid name: type: string description: type: string status: type: string scenarios: type: array items: type: object agent_definition: type: string format: uuid agent_version: type: string format: uuid evaluations: type: array items: type: object created_at: type: string format: date-time updated_at: type: string format: date-time Error: type: object properties: error: type: string securitySchemes: ApiKeyAuth: type: apiKey in: header name: X-Api-Key description: >- API Key for authentication. Click [here](https://app.futureagi.com/dashboard/keys) to access API Key SecretKeyAuth: type: apiKey in: header name: X-Secret-Key description: >- Secret Key for authentication. Click [here](https://app.futureagi.com/dashboard/keys) to access Secret Key ````