Filtering

A filter can be set for a whole thread or for a single message, and every field is optional.

Where to Set the Filter

The modelRoutingFilter lives on GenerationConfig — the same object used for model, models, responseFormat, etc. It can be set at two levels:

1. Thread-Level Default

Set via create-thread in default_generation_config.model_routing_filter. Applies to every message in the thread.

curl -X POST https://api.travila.ai/api/v1/llm/gateway/create-thread \
  -H "X-API-Key: $API_KEY" \
  -H "Authorization: Bearer $JWT" \
  -H "Content-Type: application/json" \
  -d '{
    "title": "Long research thread",
    "default_generation_config": {
      "models": [
        "google/gemini-3.6-flash:nitro",
        "anthropic/claude-sonnet-4.6:nitro",
        "anthropic/claude-sonnet-5"
      ],
      "model_routing_filter": {
        "min_context_length": 128000
      }
    }
  }'

When to use: When you want consistent capability requirements across all messages in a thread (e.g., "always require >=128k context for this long-context thread").

2. Per-Message Override

Set via send-message in override_generation_config.model_routing_filter. Applies to a single message only.

curl -X POST https://api.travila.ai/api/v1/llm/gateway/send-message \
  -H "X-API-Key: $API_KEY" \
  -H "Authorization: Bearer $JWT" \
  -H "Content-Type: application/json" \
  -d '{
    "conversation_key": "research-001",
    "user_message": {
      "role": "ROLE_USER",
      "content": [
        { "type": "CONTENT_PART_TYPE_IMAGE_BASE64", "content": "<base64_image>" },
        { "type": "CONTENT_PART_TYPE_TEXT", "content": "What is in this image?" }
      ]
    },
    "override_generation_config": {
      "models": [
        "google/gemini-3.6-flash:nitro",
        "anthropic/claude-sonnet-4.6:nitro",
        "anthropic/claude-sonnet-5"
      ],
      "model_routing_filter": {
        "required_input_modalities": ["image"]
      }
    }
  }'

When to use: When a specific message needs different model capabilities (e.g., "this message includes an image, require image input support").

The override merges — it does not replace

override_generation_config is merged field-by-field onto the config the thread would otherwise use. Send only what you want to change; everything you leave out is inherited.

The granularity is the top-level config field. A field you do set replaces the base value wholesale rather than merging into it, so a partial model_routing_filter in an override replaces the thread's filter entirely — it does not combine with it.

Two details worth knowing:

• Explicit zeros count. Scalars are presence-aware, so "temperature": 0 overrides the base rather than reading as "unset".
• Empty lists do not. Repeated and map fields have no presence, so an empty list is indistinguishable from an absent one and inherits instead. To remove tools you must send clear_tools: true; omitting tools will not do it.

ModelRoutingFilter Fields

All fields are optional. Unset/zero-value fields are ignored (no filtering on that dimension). When multiple fields are set, they are ANDed — a model must pass every specified filter.

Field	Type	Description	Behavior
min_context_length	int64	Minimum context window in tokens	Models below this are excluded
min_max_completion_tokens	int64	Minimum max output tokens	Models reporting 0 (unknown) pass through
required_input_modalities	string[]	e.g., ["image", "audio"]	Model must support ALL listed. Models with empty modalities are excluded
required_output_modalities	string[]	e.g., ["image"]	Model must support ALL listed. Models with empty modalities are excluded
max_prompt_cost	double	Max cost per prompt token (e.g., 0.000003)	0 = no limit. Unparseable pricing = pass
max_completion_cost	double	Max cost per completion token	0 = no limit. Unparseable pricing = pass
exclude_moderated	bool	Skip models with content moderation	Only filters when set to true
required_parameters	string[]	e.g., ["tools", "response_format"]	Model must support ALL listed

How filters combine

Situation	Behaviour
No model_routing_filter set	No metadata filtering. Model-ID validation still runs.
Filter set, all fields zero/unset	No filtering — the same as not setting one
Filter on override_generation_config	Applies to that message only; the next message reverts to the thread default
A single model, no models array	The filter applies to that one model. If it fails, you get an immediate error.
model plus models	The filter applies to the merged candidate list — model first, then models