> ## Documentation Index
> Fetch the complete documentation index at: https://futureagi.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# MRR (Mean Reciprocal Rank)

> Measures how early the first relevant chunk appears in the ranked retrieval results. A retrieval-stage metric for RAG pipelines that focuses on the position of the first correct answer.

<CodeGroup>
  ```python Python theme={null}
  import json
  from fi.evals import Evaluator

  evaluator = Evaluator()

  result = evaluator.evaluate(
      eval_templates="mrr",
      inputs={
          "hypothesis": json.dumps([
              "France is in Europe.",
              "Napoleon was born in Corsica.",
              "Paris is the capital of France.",
              "The Eiffel Tower was built in 1889.",
              "The Louvre is in Paris."
          ]),
          "reference": json.dumps([
              "Paris is the capital of France.",
              "The Eiffel Tower was built in 1889.",
              "The Louvre is in Paris."
          ])
      }
  )

  print(result.eval_results[0].output)   # 0.333
  print(result.eval_results[0].reason)
  ```

  ```typescript JS/TS theme={null}
  import { Evaluator } from "@future-agi/ai-evaluation";

  const evaluator = new Evaluator();

  const result = await evaluator.evaluate(
    "mrr",
    {
      hypothesis: JSON.stringify([
        "France is in Europe.",
        "Napoleon was born in Corsica.",
        "Paris is the capital of France.",
        "The Eiffel Tower was built in 1889.",
        "The Louvre is in Paris."
      ]),
      reference: JSON.stringify([
        "Paris is the capital of France.",
        "The Eiffel Tower was built in 1889.",
        "The Louvre is in Paris."
      ])
    }
  );

  console.log(result.eval_results[0]?.output);   // 0.333
  console.log(result.eval_results[0]?.reason);
  ```
</CodeGroup>

In this example, the first relevant chunk ("Paris is the capital of France.") appears at position 3, so the reciprocal rank is 1/3 = 0.333.

| **Input** |                    |          |                                                          |
| --------- | ------------------ | -------- | -------------------------------------------------------- |
|           | **Required Input** | **Type** | **Description**                                          |
|           | `hypothesis`       | `string` | JSON-serialized list of retrieved chunks in ranked order |
|           | `reference`        | `string` | JSON-serialized list of ground-truth relevant chunks     |

| **Output** |            |                                                                                          |
| ---------- | ---------- | ---------------------------------------------------------------------------------------- |
|            | **Field**  | **Description**                                                                          |
|            | **Result** | Returns a score between 0 and 1, where 1 means the first relevant chunk is at position 1 |
|            | **Reason** | Short summary string of the score, e.g. `MRR: 0.333`                                     |

<Note>
  MRR does not take a `k` parameter. It scans the entire retrieved list to find the first relevant item.
</Note>

### Batch evaluation

To evaluate multiple queries in a single call, pass a list of JSON-serialized inputs. Each element represents one retrieval evaluation:

```python Python theme={null}
results = evaluator.evaluate(
    eval_templates="mrr",
    inputs={
        "hypothesis": [
            json.dumps(["Paris is the capital of France.", "France is in Europe.", "Napoleon was born in Corsica."]),
            json.dumps(["The sky is blue.", "Water is wet."]),
            json.dumps(["Unrelated 1.", "Unrelated 2.", "Unrelated 3.", "The Louvre is in Paris."]),
        ],
        "reference": [
            json.dumps(["Paris is the capital of France.", "The Eiffel Tower was built in 1889."]),
            json.dumps(["The sky is blue.", "Water is wet."]),
            json.dumps(["The Louvre is in Paris."]),
        ],
    },
)

for i, r in enumerate(results.eval_results):
    print(f"Query {i+1}: {r.output}")
# Query 1: 1.0    (first relevant at position 1)
# Query 2: 1.0    (first relevant at position 1)
# Query 3: 0.25   (first relevant at position 4)
```

***

### How it works

MRR (Mean Reciprocal Rank) measures how quickly the retriever surfaces the first relevant result. The score is the reciprocal of the rank position where the first relevant chunk appears.

**Formula:**

```
MRR = 1 / (position of the first relevant item)
```

If the first relevant chunk is at position 1, the score is 1.0. At position 2, it's 0.5. At position 3, it's 0.333. If no relevant chunk is found, the score is 0.0.

MRR is particularly useful for question-answering RAG systems where the first relevant chunk often contains the answer. It directly measures the user experience of finding information quickly.

Matching is based on exact string equality between retrieved chunks and ground-truth chunks.

***

### What to do when MRR is Low

If MRR is low, the first relevant chunk is appearing too far down in results:

* Apply a re-ranking step to push the most relevant chunk to the top position
* Check if irrelevant but semantically similar chunks are outranking the correct answer
* Ensure query formatting matches the style of your indexed documents
* For short queries, consider query expansion to add context that helps the retriever identify the best match
* Verify that the first relevant chunk in your ground truth is actually the most directly relevant one

***

### Differentiating MRR with Similar Evals

* [**NDCG@K**](/future-agi/get-started/evaluation/builtin-evals/ndcg-at-k): NDCG\@K evaluates the ranking quality across all relevant chunks, while MRR only looks at where the first relevant chunk appears.
* [**Hit Rate**](/future-agi/get-started/evaluation/builtin-evals/hit-rate): Hit Rate is binary (was any relevant chunk retrieved?), while MRR measures exactly how high the first relevant chunk ranks.
* [**Recall@K**](/future-agi/get-started/evaluation/builtin-evals/recall-at-k): Recall\@K measures how many relevant chunks were found regardless of position, while MRR focuses solely on the position of the first relevant chunk.