Skip to main content

Keep a model on the providers you choose

Section ID: Guide/j8sto5lqv7#pin-providers

Some models are served by more than one provider. Left alone, a request for such a model may be served by a different provider from one time to the next. Name the providers you want when that consistency matters more than availability.

This page shows a preference, a restriction, where each can be set, and one limit to know before relying on either.

Before you start​

Section ID: Guide/j8sto5lqv7#before-you-start

You need a model that more than one provider serves and the provider identifiers for the ones you want. A base identifier such as together matches every variant that provider offers for the model; an identifier with a suffix such as baseten/fp8 names one variant.

The examples use deepseek/deepseek-v4.1-flash with together and baseten/fp8. Replace support-001 with your conversation's ID.

Prefer providers​

Section ID: Guide/j8sto5lqv7#prefer-providers

Set provider.order to try providers in the order you list them. This is a preference, not a guarantee: fallback stays on, so when none of the listed providers can serve the request it is routed to another provider that can. Use it when you want a usual provider but would rather get an answer than a failure.

Set the preference for one message in the body of send-message:

Request example: Send a message to a conversation · Request fields.

See Send a message to a conversation for the request and response contract. The following values illustrate the choices for this example; replace identifiers with your own authorized values.

  • conversationKey: text support-001.
  • userMessage.role: text ROLE_USER.
  • userMessage.content[0].type: text CONTENT_PART_TYPE_TEXT.
  • userMessage.content[0].content: text Summarize the last three messages from this customer..
  • overrideGenerationConfig.model: text deepseek/deepseek-v4.1-flash.
  • overrideGenerationConfig.provider.order[0]: text together.
  • overrideGenerationConfig.provider.order[1]: text baseten/fp8.

Restrict providers​

Section ID: Guide/j8sto5lqv7#restrict-providers

Set provider.only to allow only the providers you list. The request is routed with that restriction, so no other provider is eligible. If none of them can serve it, expect the run to end in a failure you can inspect, not an answer from elsewhere.

only and order can be combined: only sets who is eligible, order sets who is tried first among them.

Set the restriction for one message in the body of send-message:

Request example: Send a message to a conversation · Request fields.

See Send a message to a conversation for the request and response contract. The following values illustrate the choices for this example; replace identifiers with your own authorized values.

  • conversationKey: text support-001.
  • userMessage.role: text ROLE_USER.
  • userMessage.content[0].type: text CONTENT_PART_TYPE_TEXT.
  • userMessage.content[0].content: text Summarize the last three messages from this customer..
  • overrideGenerationConfig.model: text deepseek/deepseek-v4.1-flash.
  • overrideGenerationConfig.provider.only[0]: text together.
  • overrideGenerationConfig.provider.only[1]: text baseten/fp8.

Choose where to set it​

Section ID: Guide/j8sto5lqv7#where-to-set-it

The same provider object can be set in three places:

A per-message provider is applied over the configuration the thread would otherwise use: the active agent profile's if the thread has one, otherwise the thread default. It replaces the whole inherited provider object; it does not merge into it. Repeat any only list you still need.

Set a restriction as a thread default:

Request example: Create a new conversation thread · Request fields.

See Create a new conversation thread for the request and response contract. The following values illustrate the choices for this example; replace identifiers with your own authorized values.

  • title: text Support thread.
  • defaultGenerationConfig.model: text deepseek/deepseek-v4.1-flash.
  • defaultGenerationConfig.provider.only[0]: text together.
  • defaultGenerationConfig.provider.only[1]: text baseten/fp8.

Set a preference in an agent profile:

Request example: Create an agent profile · Request fields.

See Create an agent profile for the request and response contract. The following values illustrate the choices for this example; replace identifiers with your own authorized values.

  • profile.profileId: text support_assistant.
  • profile.name: text Support Assistant.
  • profile.description: text Order and account questions..
  • profile.whenToUse: text Use when the user asks about an order or their account..
  • profile.enabled: boolean true.
  • profile.generationConfig.model: text deepseek/deepseek-v4.1-flash.
  • profile.generationConfig.systemPrompt: text You are a support assistant. Be concise and practical..
  • profile.generationConfig.provider.order[0]: text together.
  • profile.generationConfig.provider.order[1]: text baseten/fp8.

Know the fallback limit​

Section ID: Guide/j8sto5lqv7#fallback-limit

Setting allowFallbacks to false does not turn fallback off today; the request is still sent with fallback allowed. To keep a request inside a set of providers, use provider.only.

The API does not currently return which provider served a request.

Section ID: Guide/j8sto5lqv7#related

Document ID: Guide/j8sto5lqv7