Dataset identity and run provenance
Section ID: Guide/jecche7l32#overview
An evaluation harness executes cases and preserves their inputs. Dataset identities, recorded run associations and enriched read models describe different parts of that evidence.
Dataset and run identifiers
Section ID: Guide/jecche7l32#dataset-identifiers
| Operation context | Dataset identity |
|---|---|
List dataset items or list dataset runs: datasetName | Human-readable dataset name. |
List dataset items: datasetId | Actual dataset ID returned by the API. |
Add dataset item: datasetId | Dataset name, despite the field name. |
Update dataset item: datasetId | Dataset name, despite the field name. |
Follow the selected operation's contract; there is no global substitution between a dataset name and ID.
Run association and scoring limits
Section ID: Guide/jecche7l32#run-association
Your harness executes the cases under its configured permissions and budget; the platform does not execute a dataset. A turn joins a dataset run only when the message that starts it carries an experiment block on send-message or send-message-sync, naming the dataset ID (not the name), the run name and, per case, the dataset item ID. There is no operation that associates an already-recorded trace with a run. Turns that carry a dataset item ID and turns that do not currently land in different runs even under the same run name, so set it on every turn of a run. Read back the run and expected items before treating the experiment as recorded; keep your harness manifest and item outcomes, and contact your Travila operator if an expected run remains absent.
A run item links a dataset item to the trace produced by the harness, optionally narrowed to an observation. It does not itself carry a score. The current public record-score operation supports qualified trace scoring; EVAL_TARGET_TYPE_DATASET_RUN appearing in the schema does not establish working dataset-run scoring. Keep harness results in a versioned report and use only verified score-ingestion paths until that gap is closed.
profileRevisionHash is supplied run metadata. It is not the same as the producer's current configHash, and neither alone recreates an experiment. Save immutable profile and fragment revisions, resolved settings, model/provider, code, tool fixtures, memory/index versions, dataset item versions, evaluator/rubric version and environment. Frozen inputs must be explicitly captured; the current add-item endpoint does not capture them for you.
Run reads and incomplete enrichment
Section ID: Guide/jecche7l32#run-reading
| Read behavior | Interpretation |
|---|---|
| Current run-list pagination | Continue with cursorPage.nextCursor while preserving dataset and time bounds. Older deployments may expose page-based pagination. |
| Dataset-name lookup | Searches only the first dataset page. |
| Item-count enrichment | Can be capped or fail; zero does not prove that no cases ran. |
lastRunAt and runCount | History hints that can remain absent or zero after failed enrichment. |
Read the expected run directly and compare it with the harness record before concluding that the suite has not been executed.
Cursor pagination
Section ID: Guide/jecche7l32#pagination
Start without a cursor, then pass the exact cursorPage.nextCursor returned by the previous response. Keep the original scope, filters and time bounds. Continue until no cursor remains; a short or empty page alone does not finish the search.
Preserve your collected identities and known coverage if retrieval is interrupted. Do not infer page numbers or totals from a cursor, treat a missing total as zero, or treat sampled facets as every value in the account. Large integer fields use decimal strings; preserve their precision as described in the common API contract.
Document ID: Guide/jecche7l32