> ## Documentation Index
> Fetch the complete documentation index at: https://futureagi.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# AI Evaluation SDK Cookbooks

> Hands-on tutorials that solve real problems you will face when building, testing, and securing AI applications with the fi-evals Python SDK.

These cookbooks walk you through the AI Evaluation SDK from zero to production. Each one solves a concrete problem -- hallucination detection, RAG debugging, prompt injection defense, streaming safety, and more -- with runnable code you can copy straight into your project.

## Prerequisites

```bash theme={null}
pip install ai-evaluation
```

Cookbooks that use an LLM judge (02, 08, 09) require a `GOOGLE_API_KEY` for Gemini. All other cookbooks run entirely locally with no API keys and no network calls.

## Cookbooks

<CardGroup cols={2}>
  <Card title="01 - Local Metrics" icon="gauge" href="/cookbook/ai-evaluation/local-metrics">
    Catch hallucinations, wrong dosages, and contradictions in a medical chatbot -- all locally in under one second.
  </Card>

  <Card title="02 - LLM-as-Judge" icon="scale-balanced" href="/cookbook/ai-evaluation/llm-judge">
    When heuristics miss paraphrases, use Gemini to judge accuracy with augment=True and custom prompts.
  </Card>

  <Card title="03 - RAG Evaluation" icon="magnifying-glass-chart" href="/cookbook/ai-evaluation/rag-evaluation">
    Diagnose where your RAG pipeline fails -- is retrieval pulling the wrong docs, or is the LLM hallucinating?
  </Card>

  <Card title="04 - Guardrails" icon="shield-halved" href="/cookbook/ai-evaluation/guardrails">
    Build a sub-10ms security middleware that blocks jailbreaks, code injection, PII leaks, and secret exposure.
  </Card>

  <Card title="05 - Streaming Safety" icon="bolt" href="/cookbook/ai-evaluation/streaming">
    Monitor streaming LLM output token-by-token and cut the stream the moment safety thresholds are breached.
  </Card>

  <Card title="06 - AutoEval" icon="wand-magic-sparkles" href="/cookbook/ai-evaluation/autoeval">
    Describe your app in plain English and get an auto-configured test pipeline you can export to CI/CD.
  </Card>

  <Card title="07 - Feedback Loop" icon="arrows-rotate" href="/cookbook/ai-evaluation/feedback-loop">
    Store developer corrections in ChromaDB and teach your LLM judge to stop repeating the same mistakes.
  </Card>

  <Card title="08 - Multimodal Judge" icon="image" href="/cookbook/ai-evaluation/multimodal-judge">
    Pass images and audio to the LLM judge -- verify product descriptions match photos, check transcription accuracy.
  </Card>
</CardGroup>

## Progression

The cookbooks are designed to build on each other:

| Stage                    | Cookbook            | What you add                               |
| ------------------------ | ------------------- | ------------------------------------------ |
| **Start here**           | 01 Local Metrics    | Fast, free, local checks                   |
| **Improve accuracy**     | 02 LLM-as-Judge     | LLM refinement for hard cases              |
| **Debug your RAG**       | 03 RAG Evaluation   | Separate retrieval vs. generation failures |
| **Secure inputs**        | 04 Guardrails       | Block attacks before they reach the LLM    |
| **Protect in real-time** | 05 Streaming        | Cut off toxic output mid-stream            |
| **Automate setup**       | 06 AutoEval         | Generate pipelines from descriptions       |
| **Learn from mistakes**  | 07 Feedback Loop    | Few-shot feedback for the judge            |
| **Go multimodal**        | 08 Multimodal Judge | Images and audio evaluation                |