Skip to main content

​
What it is

A score is the atomic data record created every time an annotation label is applied to a source entity. It is the unified annotation primitive in FutureAGI, replacing the legacy TraceAnnotation model with a single structure that works identically across traces, spans, sessions, dataset rows, prototype runs, and simulation executions. Every score answers five questions: what was annotated (source), how it was annotated (label and value), who annotated it (annotator), when (timestamps), and why (optional notes and queue context).

​
Score fields

​
Source types

Scores can target any of the following entity types. The source_type and source_id fields together form a polymorphic foreign key to the annotated entity.

​
Two ways to create scores

​
Queue workflow (managed)

Scores created through an annotation queue are linked to a queue item via the queue_item field. The queue manages assignment, progress tracking, and completion logic.
1

Add items to a queue

Select entities (traces, dataset rows, etc.) in their respective views and click Add to Queue. Each becomes a queue item.
2

Annotate through the workspace

Click Start Annotating on the queue detail page. The workspace presents items one at a time with the queue’s labels. Each submitted annotation creates a score.
3

Auto-completion

When all labels are scored by the required number of annotators, the queue item auto-completes.

​
Inline annotation (direct)

Scores can also be created directly from the detail view of any supported entity — without going through a queue. Inline annotations have queue_item set to null.
  • Trace detail: Open a trace, expand the annotation panel, and apply any label from your organization.
  • Session grid: Annotation columns appear directly in the sessions table for quick scoring.
  • Dataset view: Annotate individual rows from the dataset table.
Inline annotations are ideal for ad-hoc feedback during investigation or review. They produce the same Score records and appear alongside queue-created scores in all views and exports.

​
Value formats by label type

The value field in a score is JSON. Its shape depends on the label type:

​
Where scores appear

Scores are surfaced everywhere the annotated entity is displayed:
  • Trace detail view — Annotation panel shows all scores for the trace and its spans.
  • Sessions grid — Dynamic annotation columns display score values inline with session data. Filter and sort by annotation values.
  • Dataset table — Score values appear as columns alongside dataset row data.
  • Queue detail — The items tab shows all scores submitted for each queue item.
  • API — Query scores programmatically with filters on source type, label, annotator, and date range.

​
Bidirectional sync

Scores created through different paths stay synchronized:
  • An annotation submitted on a trace via Observe automatically creates a corresponding score visible in any queue containing that trace.
  • A score submitted through a queue workflow is immediately visible in the trace detail and session grid views.
This ensures a single source of truth regardless of where the annotation originated.

​
What you can do next

Annotation Labels

Learn about the five label types that define score value formats.

Queues & Workflow

Understand how queues manage the annotation lifecycle and produce scores.

Quickstart

Walk through the full flow from label creation to submitted scores.