Reading Traces

The read side of the API: finding turns, opening one up, and pulling the scores already attached to them. Everything here is non-destructive and works with a bare sk_… key — no user identity needed.

Read the overview first for how a turn is addressed and why trace content is redacted.

Find Traces

One trace is one turn, however many tool-call rounds the agent went through.

curl -X POST https://api.travila.ai/api/v1/evals/list-traces \
  -H "X-API-Key: sk_your_key_here" \
  -H "Content-Type: application/json" \
  -d '{
    "profileId": "nutrition_coach",
    "fromTime": "2026-08-01T00:00:00Z",
    "toTime": "2026-08-14T00:00:00Z",
    "level": "OBSERVATION_LEVEL_ERROR",
    "page": 1,
    "pageSize": 25
  }'

Response:

{
  "traces": [
    {
      "traceId": "trc_a1b2c3",
      "name": "generation",
      "conversationId": "conv_123",
      "sourceUserMessageId": "msg_abc",
      "sessionId": "sess_def",
      "profileId": "nutrition_coach",
      "configHash": "9f2c1e…",
      "userId": "user_123",
      "environment": "production",
      "timestamp": "2026-08-12T14:02:11Z",
      "latencySeconds": 8.41,
      "inputTokens": "3120",
      "outputTokens": "412",
      "totalTokens": "3532",
      "totalCost": 0.0041,
      "observationCount": 6,
      "errorCount": 1,
      "scores": [{"name": "helpfulness", "numericValue": 4, "source": "SCORE_SOURCE_HUMAN"}]
    }
  ],
  "page": {"page": 1, "limit": 25, "totalItems": 87, "totalPages": 4},
  "contentRedacted": true
}

The narrowing fields are conversationId, profileId, userId, sessionId, environment, release, version, tags, name, level, and the fromTime / toTime window. searchQuery with searchType does free-text search, and orderBy changes the sort.

Token counts are 64-bit and therefore strings; counts like observationCount and errorCount are 32-bit and arrive as numbers.

errorCount, warningCount, defaultCount and debugCount summarize the observation levels inside the trace — a cheap way to find broken turns without opening each one.

Structured filters

For anything the named fields do not cover, filters takes a list of column predicates:

{
  "filters": [
    {
      "column": "totalCost",
      "operator": ">",
      "type": "EVAL_FILTER_TYPE_NUMBER",
      "numberValue": 0.05
    }
  ],
  "page": 1
}

Each entry names a column, an operator, a type, and the value field matching that type — stringValue, numberValue, booleanValue, timeValue or stringValues. Setting the wrong one for the declared type silently filters on nothing.

Open One Trace

curl -X POST https://api.travila.ai/api/v1/evals/get-trace \
  -H "X-API-Key: sk_your_key_here" \
  -H "Content-Type: application/json" \
  -d '{
    "conversationId": "conv_123",
    "sourceUserMessageId": "msg_abc"
  }'

Pass exactly one address: either traceId, or the conversationId + sourceUserMessageId pair. The pair is what callers usually hold — see Addressing a turn.

Response:

{
  "trace": {
    "summary": {"traceId": "trc_a1b2c3", "name": "generation", "latencySeconds": 8.41},
    "observations": [
      {
        "observationId": "obs_1",
        "name": "memory-search",
        "type": "OBSERVATION_TYPE_SPAN",
        "level": "OBSERVATION_LEVEL_DEFAULT",
        "startTime": "2026-08-12T14:02:11Z",
        "endTime": "2026-08-12T14:02:11.940Z",
        "latencySeconds": 0.94
      },
      {
        "observationId": "obs_2",
        "name": "generation",
        "type": "OBSERVATION_TYPE_GENERATION",
        "model": "google/gemini-3.6-flash",
        "inputTokens": "3120",
        "outputTokens": "412",
        "inputCost": 0.0031,
        "outputCost": 0.001,
        "level": "OBSERVATION_LEVEL_DEFAULT"
      }
    ]
  },
  "contentRedacted": true
}

The observations array is the turn's internals — memory lookups, tool calls, and the generations themselves, each with its own timing, cost and level. This is where a slow turn's time actually went.

inputJson / outputJson / metadataJson are JSON encoded as strings; parse them rather than reading them as objects.

Observations Across Traces

Same records, queried across traces rather than within one — for questions like "which tool fails most often" or "how slow is memory-search this week".

curl -X POST https://api.travila.ai/api/v1/evals/list-observations \
  -H "X-API-Key: sk_your_key_here" \
  -H "Content-Type: application/json" \
  -d '{
    "filters": [
      {
        "column": "level",
        "operator": "=",
        "type": "EVAL_FILTER_TYPE_STRING",
        "stringValue": "ERROR"
      }
    ],
    "page": 1,
    "pageSize": 50
  }'

Response:

{
  "observations": [
    {
      "observationId": "obs_9",
      "traceId": "trc_a1b2c3",
      "parentObservationId": "obs_2",
      "name": "get_weather",
      "level": "OBSERVATION_LEVEL_ERROR",
      "statusMessage": "upstream timeout",
      "latencySeconds": 30.0
    }
  ],
  "page": {"page": 1, "limit": 50, "totalItems": 12, "totalPages": 1},
  "contentRedacted": true
}

parentObservationId nests observations inside one another, so a tool call sits under the generation that requested it.

Sessions

A session groups conversations that belong to one stretch of user activity — the unit to read when a single turn looks fine but the overall experience did not.

curl -X POST https://api.travila.ai/api/v1/evals/list-sessions \
  -H "X-API-Key: sk_your_key_here" \
  -H "Content-Type: application/json" \
  -d '{
    "userId": "user_123",
    "fromTime": "2026-08-01T00:00:00Z",
    "page": 1,
    "pageSize": 25
  }'

Response:

{
  "sessions": [
    {
      "sessionId": "sess_def",
      "conversationIds": ["conv_123", "conv_456"],
      "userIds": ["user_123"],
      "environment": "production",
      "createdAt": "2026-08-12T13:58:02Z",
      "durationSeconds": 412.7,
      "traceCount": 9,
      "inputTokens": "18220",
      "outputTokens": "3901",
      "commentCount": 2,
      "scores": [{"name": "satisfaction", "numericValue": 3, "source": "SCORE_SOURCE_USER"}]
    }
  ],
  "page": {"page": 1, "limit": 25, "totalItems": 3, "totalPages": 1}
}

Then open one to get its traces alongside the summary:

curl -X POST https://api.travila.ai/api/v1/evals/get-session \
  -H "X-API-Key: sk_your_key_here" \
  -H "Content-Type: application/json" \
  -d '{"sessionId": "sess_def"}'

Response:

{
  "session": {
    "summary": {"sessionId": "sess_def", "durationSeconds": 412.7, "traceCount": 9},
    "traces": [
      {"traceId": "trc_a1b2c3", "name": "generation", "latencySeconds": 8.41}
    ]
  },
  "contentRedacted": true
}

traces here carries the same TraceSummary shape as list-traces, so a session view and a trace list can share rendering code.

Read Scores

Scores recorded by judges, reviewers, harnesses and end users all land in one place.

curl -X POST https://api.travila.ai/api/v1/evals/list-scores \
  -H "X-API-Key: sk_your_key_here" \
  -H "Content-Type: application/json" \
  -d '{
    "conversationId": "conv_123",
    "dataType": "SCORE_DATA_TYPE_NUMERIC",
    "page": 1,
    "pageSize": 50
  }'

Response:

{
  "scores": [
    {
      "scoreId": "scr_7788",
      "name": "helpfulness",
      "dataType": "SCORE_DATA_TYPE_NUMERIC",
      "numericValue": 4,
      "source": "SCORE_SOURCE_HUMAN",
      "authorUserId": "reviewer_9",
      "configId": "cfg_help",
      "traceId": "trc_a1b2c3",
      "createdAt": "2026-08-12T16:20:00Z",
      "comment": "Answered, but buried the actual number."
    }
  ],
  "page": {"page": 1, "limit": 50, "totalItems": 1, "totalPages": 1}
}

Filter by conversationId, dataType, source, name, configId, traceId, sessionId, datasetRunId, queueId, or a structured filters list. Filtering by configHash is how you answer "did quality move when we changed the prompt?" — see the overview.

source tells you who produced it and the values are not interchangeable — see Never blend score sources before averaging across them.

The Overview Dashboard

One aggregate call behind a quality dashboard: volumes, score averages by profile, latency percentiles, and the worst recent scores.

curl -X POST https://api.travila.ai/api/v1/evals/get-overview \
  -H "X-API-Key: sk_your_key_here" \
  -H "Content-Type: application/json" \
  -d '{
    "fromTime": "2026-08-01T00:00:00Z",
    "toTime": "2026-08-14T00:00:00Z",
    "environment": "production"
  }'

Response:

{
  "overview": {
    "totalTraces": "18422",
    "annotatedTraceCount": "310",
    "scoredTraceRatio": 0.017,
    "byProfile": [
      {
        "profileId": "nutrition_coach",
        "traceCount": "9120",
        "avgJudgeScore": 4.1,
        "avgEndUserRating": 3.8,
        "thumbsDownCount": "42"
      }
    ],
    "latencyPercentiles": [
      {"traceName": "generation", "p50": 6.2, "p90": 11.4, "p95": 14.9, "p99": 28.1}
    ],
    "tracesOverTime": [
      {"bucket": "2026-08-12T00:00:00Z", "series": "traces", "value": 1420}
    ],
    "recentLowScores": [
      {"scoreId": "scr_7788", "name": "helpfulness", "numericValue": 1, "traceId": "trc_a1b2c3"}
    ]
  }
}

Pass profileId to scope the whole dashboard to one profile. Counts are 64-bit and arrive as strings; averages and ratios are plain numbers.

scoredTraceRatio is the honest one to watch: an average judge score of 4.1 means little if only 1.7% of traces were ever scored.

Find Traces​

Structured filters​

Open One Trace​

Observations Across Traces​

Sessions​

Read Scores​

The Overview Dashboard​

Find Traces

Structured filters

Open One Trace

Observations Across Traces

Sessions

Read Scores

The Overview Dashboard