> ## Documentation Index
> Fetch the complete documentation index at: https://futureagi.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.
# Queues & Workflow
> Learn how annotation queues organize work -- statuses, assignment strategies, reservations, multi-annotator support, and review workflows.
## What it is
An annotation queue is a managed campaign that groups items to annotate, assigns them to annotators, tracks progress, and enforces quality controls. Queues sit between labels (what to measure) and scores (the resulting data), providing the operational layer that turns annotation from an ad-hoc activity into a structured workflow.
## Queue lifecycle
A queue moves through a defined set of statuses:
| Transition | Description |
| ----------------------------------- | ------------------------------------------------------------------------------------------------ |
| **Draft** → **Active** | Start accepting annotations. Annotators can begin work. |
| **Active** → **Paused** | Temporarily stop annotation. No new items can be picked up, but in-progress items are preserved. |
| **Paused** → **Active** | Resume annotation. |
| **Active / Paused** → **Completed** | All items are done, or you manually close the queue. |
| **Completed** → **Active** | Re-open the queue if new items are added or you need additional annotations. |
## Item statuses
Each item in a queue has its own status:
| Status | Meaning |
| ------------------ | -------------------------------------------------------------------------------------------------- |
| **Pending** | Waiting for an annotator to pick it up. |
| **In Progress** | An annotator has opened the item and is actively annotating. |
| **Completed** | All required annotations have been submitted. |
| **Skipped** | An annotator chose to skip this item. It remains available for others. |
| **Pending Review** | Annotations are done but the item is awaiting reviewer approval (when review workflow is enabled). |
## Assignment strategies
Assignment strategies control how items are distributed to annotators when they click **Start Annotating**.
| Strategy | Behavior | Best For |
| ----------------- | ---------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------- |
| **Manual** | Annotators browse and pick items themselves from the queue list. | Small queues or exploratory annotation where annotators need context to choose. |
| **Round Robin** | Items are distributed cyclically across annotators in order. Each annotator gets the next available item in rotation. | Even distribution when annotators work at similar speeds. |
| **Load Balanced** | Items are distributed based on each annotator's current workload. Annotators with fewer in-progress items get the next item. | Teams with varying availability or part-time annotators. |
## Reservation system
When an annotator opens an item, the system **reserves** it for a configurable timeout period. This prevents two annotators from working on the same item simultaneously.
* **Default timeout**: 1 hour.
* **Configurable range**: 15 minutes to 4 hours.
* **Expiry behavior**: If the annotator does not submit or skip within the timeout, the reservation expires and the item returns to **Pending** status for another annotator to pick up.
Reservations are visible in the queue detail view so managers can monitor active work.
## Multi-annotator support
For tasks that benefit from agreement between multiple reviewers, set the **Annotations Required** field (1-10) when creating or editing a queue.
* Each item must receive the configured number of complete annotations before it transitions to **Completed**.
* Different annotators independently annotate the same item -- they do not see each other's responses.
* The queue analytics tab shows inter-annotator agreement metrics once multiple annotators have scored the same items.
An item is considered fully annotated by a single annotator only when all labels attached to the queue have been scored. Partial submissions are saved but do not count toward the required annotation count.
## Review workflow
Enable **Requires Review** on a queue to add a review step after annotation:
Annotators complete their work as usual. When all required annotations are submitted, the item moves to **Pending Review** instead of **Completed**.
A designated reviewer opens the item, sees all submitted annotations, and either **Approves** (moves to Completed) or **Rejects** (sends back to Pending for re-annotation).
This is useful for high-stakes labeling tasks where a senior reviewer must validate annotations before they become final.
## Guidelines
Each queue supports a **Guidelines** field -- a markdown-formatted instruction document shown to annotators when they open the annotation workspace. Use guidelines to:
* Define the annotation criteria and edge cases.
* Provide examples of correct and incorrect annotations.
* Specify when to skip an item.
* Link to external reference material.
Well-written guidelines significantly improve annotation consistency, especially with larger teams.
## Auto-completion
Items auto-complete when the following conditions are met:
1. All labels attached to the queue have been scored for the item.
2. The required number of annotators (set by **Annotations Required**) have each fully annotated the item.
3. If **Requires Review** is enabled, the reviewer has approved the item.
No manual intervention is needed -- the system tracks progress and transitions items automatically.
## Creating a queue
Navigate to **Annotations** in the left sidebar and select the **Queues** tab. Click **Create Queue**.
Fill in the queue settings:
| Field | Description |
| -------------------- | ----------------------------------------------------- |
| Name | A descriptive name for the campaign. |
| Labels | Select one or more labels that annotators will apply. |
| Assignment Strategy | Manual, Round Robin, or Load Balanced. |
| Annotators | Add team members who will annotate. |
| Annotations Required | Number of independent annotators per item (1-10). |
| Reservation Timeout | How long an item stays reserved for one annotator. |
| Requires Review | Whether completed items need reviewer approval. |
| Guidelines | Markdown instructions for annotators. |
Items can be added from Observe (traces, spans, sessions), Datasets (rows), Prototypes (runs), or Simulations (executions). Select items in their respective views and click **Add to Queue**.
## What you can do next
Learn about the five label types you can attach to queues.
Understand the data model behind every annotation.
Walk through the full annotation flow end to end in 5 minutes.