Chat Completions API

POST /api/v1/chat/completions — the OpenAI-compatible chat endpoint. Send messages, get a completion, across every model in the catalog.

Send messages to a model and get a completion. A drop-in replacement for OpenAI's POST /v1/chat/completions: point your OpenAI client at the AnyRouter base URL.

Authenticate with an LLM API key (sk-ar-v1-…) from /dashboard/keys. Management keys (ak_…) are not accepted.

Request

FieldTypeRequiredDescription
modelstringyesprovider/model id (e.g. openai/gpt-5.4-mini). See List models.
messagesarrayyesOrdered conversation. Each message has role (system / user / assistant / tool) and content.
temperaturenumberno0–2. Lower is more deterministic. Default: 1.
top_pnumberno0–1. Nucleus sampling. Default: 1.
max_tokensintegernoUpper bound on output tokens.
streambooleannoIf true, returns a streaming SSE response.
stopstring | string[]noStop sequences.
toolsarraynoFunction-calling / tool-use spec.
tool_choicestring | objectnoForce a specific tool or auto.
response_formatobjectno{ "type": "json_object" } for guaranteed JSON.
seedintegernoDeterministic sampling seed (provider-dependent).
userstringnoStable end-user identifier for abuse monitoring.
session_idstringnoSticky-session id (≤256 chars) that groups related requests in the Request Logs dashboard. Also settable via the x-session-id header; the body field wins if both are sent.
providerobjectnoRequest-level routing preferences such as only, ignore, order, sort, allow_fallbacks, and max_price. See Smart Routing.
traceobjectnoOptional tracing metadata. Supports trace_id, trace_name, span_name, generation_name, and parent_span_id.

Force a provider per request with X-AnyRouter-Provider: openai, or a fallback list such as X-AnyRouter-Provider: groq,openai. Every response returns X-Request-ID and X-AnyRouter-Provider (the upstream that served it).

Routing preferences

provider constrains upstream selection without changing the model id:

json
{
  "provider": {
    "only": ["OpenAI", "Groq"],
    "order": ["Groq", "OpenAI"],
    "sort": "ttft",
    "allow_fallbacks": true,
    "max_price": {
      "prompt": "1.00",
      "completion": "2.00"
    }
  }
}

AnyRouter tracks live TTFT and TPS per upstream and re-ranks on every request.

provider.sortSorts byDirectionWhen to use
costListed input price per million tokensLowest price firstHigh-volume, cost-sensitive work
ttftMedian time-to-first-token (ms)Lowest latency firstLatency-sensitive workloads
tpsMedian tokens-per-second throughputHighest firstLong-output generation

Shortcut suffixes on the model id pick the sort for you: model:floor → cost, model:fast → ttft, model:nitro → tps. price, latency and throughput are aliases for cost, ttft and tps. Supported fields: only, ignore, order, sort, allow_fallbacks, max_price.prompt, max_price.completion, preferred_max_latency, preferred_min_throughput; others are accepted but ignored.

Response

json
{
  "id": "chatcmpl-8yWq4JqLfEjJ9L",
  "object": "chat.completion",
  "created": 1760000000,
  "model": "anthropic/claude-sonnet-4-6",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "NAT maps many private IP addresses to one public IP so devices behind a router can share a single internet connection."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 28,
    "completion_tokens": 30,
    "total_tokens": 58,
    "cost": 0.000342
  }
}
finish_reasonMeaning
stopModel completed naturally or hit a stop sequence.
lengthHit max_tokens.
tool_callsModel invoked one or more tools; caller must run them and resume.
content_filterUpstream safety filter triggered.
errorUpstream provider error; see error on the response.

Cost of a request

usage.cost is the USD charged to your credits, 0 when nothing was charged (a free model or your own provider key, BYOK).

FieldTypeDescription
usage.costnumberUSD charged for this request. 0 for free models and BYOK.
usage.cost_details.upstream_inference_costnumberUSD the serving provider reported for the call, when it reports one. Present only when known.

Where it appears:

  • Non-streaming: on every response that includes usage.
  • Streaming: with stream_options.include_usage: true, on the one usage chunk (an empty choices array) that follows the finish chunk, next to anyrouter_metadata. Without it, no chunk carries usage, as in OpenAI's API. The charge is then anyrouter_metadata.cost.amount on the finish chunk, which is sent once the provider ends the stream. anyrouter_metadata.cost.amount always equals usage.cost, on streaming and non-streaming responses alike. Either way each choice sends a non-null finish_reason exactly once, and nothing but data: [DONE] follows the last chunk.
  • /responses: usage.cost is the charged amount on non-streaming responses and on the terminal response.completed event of a stream. The input/output split in usage.cost_details is not returned, because it never added up to the charge for cached tokens or non-list prices. /messages reports the charged amount as usage.cost too: on the response, or on the final message_delta of a stream (Messages).

It matches your usage history. Once a chunk shows usage.cost, that amount is charged even if the client or provider disconnects before the stream ends.

Examples

bash
curl https://anyrouter.dev/api/v1/chat/completions \
  -H "Authorization: Bearer sk-ar-your-key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic/claude-sonnet-4-6",
    "messages": [
      { "role": "system", "content": "You are a concise assistant." },
      { "role": "user", "content": "Explain NAT in one sentence." }
    ],
    "max_tokens": 200
  }'

Tool use

Pass a tools array with function schemas, then handle any returned tool_calls:

bash
curl https://anyrouter.dev/api/v1/chat/completions \
  -H "Authorization: Bearer sk-ar-your-key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-5.4-mini",
    "messages": [
      { "role": "user", "content": "What is the weather in Tokyo?" }
    ],
    "tools": [
      {
        "type": "function",
        "function": {
          "name": "get_weather",
          "description": "Get the current weather for a city",
          "parameters": {
            "type": "object",
            "properties": { "city": { "type": "string" } },
            "required": ["city"]
          }
        }
      }
    ]
  }'

See Tool use.

Errors

StatusMeaningFix
400Malformed body or invalid parameters.Check the request against the field table above.
401Missing or invalid API key.Send a valid sk-ar-… key as Authorization: Bearer.
402Out of credits or upstream BYOK billing failure.Top up credits, or read metadata.upstream_message.
404Unknown model.Verify the id with List models.
429Rate limit hit.Back off; inspect X-RateLimit-* response headers.
502Upstream 5xx or every fallback failed.Retry; see metadata.upstream_attempts.

See Errors for the full status table and retry recipes.

AnyRouter routes requests to third-party models. Models and outputs are provided as is, with no warranty; you use them at your own risk. Disclaimer