Model Fallbacks

Configure automatic failover between AI models when providers are down, rate-limited, or return retryable errors.

List backup models in a models array and AnyRouter tries them when the primary model's providers fail. Provider routing fails over between upstreams of one model; model fallbacks fail over between different models.

Only /api/v1/chat/completions reads models. Responses and Messages do not yet.

Use it

Send ids from the catalog in priority order. The primary model goes first, then each models entry in order. A duplicate of model is dropped, and entries with no configured upstream are skipped silently.

curl https://anyrouter.dev/api/v1/chat/completions \
  -H "Authorization: Bearer <ANYROUTER_API_KEY>" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic/claude-sonnet-4-6",
    "models": ["openai/gpt-5.4-mini", "google/gemini-3.1-flash"],
    "messages": [{"role": "user", "content": "What is the meaning of life?"}]
  }'

The OpenAI SDK has no top-level models field. In Python pass extra_body={"models": [...]}; in TypeScript add models with a // @ts-expect-error on the line above.

Behavior

  • Each attempt has a 60-second timeout. The next candidate is tried on a timeout, an abort, a network error, or one of 402, 403, 404, 408, 425, 429, 500, 502, 503, 504. Other statuses, such as 400 context-length errors, return immediately.
  • Within each model, normal provider routing applies: every eligible upstream is tried before the next model. Set provider.allow_fallbacks: false with provider.order to limit a model to one provider.
  • Fallback only happens on the initial response. Once the upstream returns 200 and SSE bytes start flowing, mid-stream failures surface as-is.

Pricing

You are billed for the model that answered. The response model field (and the metadata frame of a streamed response) is rewritten to that model id.

  • usage.cost: what AnyRouter charged (0 for free models and BYOK).
  • usage.cost_details.upstream_inference_cost: the upstream-reported cost, when available.
  • anyrouter_metadata.cost.upstreamAmount: the same on the streamed metadata frame, when available.

Sort across all models

To pick the best endpoint across every model instead of trying the primary first, combine models with partition: "none". See Advanced Sorting with Partition.

{ "provider": { "sort": { "by": "throughput", "partition": "none" } } }

Troubleshooting

SymptomFix
Fallback never ranThe failure was not retryable (for example a 400). Fix the request.
A listed model is ignoredIt has no configured upstream. Check the id in Models.
Stream failed midway with no fallbackFallback covers the initial response only. Retry on the client.

AnyRouter routes requests to third-party models. Models and outputs are provided as is, with no warranty; you use them at your own risk. Disclaimer