Dataset identity and run provenance
An evaluation harness executes cases and preserves their inputs. Dataset identities, recorded run associations and enriched read models describe different parts of that evidence.
Dataset and run identifiers
| Operation context | Dataset identity |
|---|---|
List dataset items or list dataset runs: datasetName | Human-readable dataset name. |
List dataset items: datasetId | Actual dataset ID returned by the API. |
Add dataset item: datasetId | Dataset name, despite the field name. |
Update dataset item: datasetId | Dataset name, despite the field name. |
Follow the selected operation's contract; there is no global substitution between a dataset name and ID.
Run association and scoring limits
Your harness executes the cases under its configured permissions and budget.
record-dataset-run does not execute them. This operation can associate existing
traces only where that evaluation workflow is supported for your account. Other
configurations can return HTTP 200 without creating the associations. Read back
the run and expected items before treating the experiment as recorded.
This reference does not yet document an experiment-context input on message requests. Keep your harness manifest and item outcomes when association is unavailable; contact your Travila operator if an expected run remains absent.
A run item links a dataset item to the trace produced by the harness, optionally narrowed to an observation. It does not itself carry a score. The current public record-score operation supports qualified trace scoring; EVAL_TARGET_TYPE_DATASET_RUN appearing in the schema does not establish working dataset-run scoring. Keep harness results in a versioned report and use only verified score-ingestion paths until that gap is closed.
profileRevisionHash is supplied run metadata. It is not the same as the producer's current configHash, and neither alone recreates an experiment. Save immutable profile and fragment revisions, resolved settings, model/provider, code, tool fixtures, memory/index versions, dataset item versions, evaluator/rubric version and environment. Frozen inputs must be explicitly captured; the current add-item endpoint does not capture them for you.
Run reads and incomplete enrichment
| Read behavior | Interpretation |
|---|---|
| Current run-list pagination | Continue with cursorPage.nextCursor while preserving dataset and time bounds. Older deployments may expose page-based pagination. |
| Dataset-name lookup | Searches only the first dataset page. |
| Item-count enrichment | Can be capped or fail; zero does not prove that no cases ran. |
lastRunAt and runCount | History hints that can remain absent or zero after failed enrichment. |
Read the expected run directly and compare it with the harness record before concluding that the suite has not been executed.
Cursor pagination
Start without a cursor, then pass the exact cursorPage.nextCursor returned by the previous response. Keep the original scope, filters and time bounds. Continue until no cursor remains; a short or empty page alone does not finish the search.
Preserve your collected identities and known coverage if retrieval is interrupted. Do not infer page numbers or totals from a cursor, treat a missing total as zero, or treat sampled facets as every value in the account. Large integer fields use decimal strings; preserve their precision as described in the common API contract.