Chat Completions API
POST /api/v1/chat/completions — the OpenAI-compatible chat endpoint. Send messages, get a completion, across every model in the catalog.
Send messages to a model and get a completion. A drop-in replacement for OpenAI's POST /v1/chat/completions: point your OpenAI client at the AnyRouter base URL.
Authenticate with an LLM API key (sk-ar-v1-…) from /dashboard/keys. Management keys (ak_…) are not accepted.
Request
| Field | Type | Required | Description |
|---|---|---|---|
model | string | yes | provider/model id (e.g. openai/gpt-5.4-mini). See List models. |
messages | array | yes | Ordered conversation. Each message has role (system / user / assistant / tool) and content. |
temperature | number | no | 0–2. Lower is more deterministic. Default: 1. |
top_p | number | no | 0–1. Nucleus sampling. Default: 1. |
max_tokens | integer | no | Upper bound on output tokens. |
stream | boolean | no | If true, returns a streaming SSE response. |
stop | string | string[] | no | Stop sequences. |
tools | array | no | Function-calling / tool-use spec. |
tool_choice | string | object | no | Force a specific tool or auto. |
response_format | object | no | { "type": "json_object" } for guaranteed JSON. |
seed | integer | no | Deterministic sampling seed (provider-dependent). |
user | string | no | Stable end-user identifier for abuse monitoring. |
session_id | string | no | Sticky-session id (≤256 chars) that groups related requests in the Request Logs dashboard. Also settable via the x-session-id header; the body field wins if both are sent. |
provider | object | no | Request-level routing preferences such as only, ignore, order, sort, allow_fallbacks, and max_price. See Smart Routing. |
trace | object | no | Optional tracing metadata. Supports trace_id, trace_name, span_name, generation_name, and parent_span_id. |
Force a provider per request with X-AnyRouter-Provider: openai, or a fallback list such as X-AnyRouter-Provider: groq,openai. Every response returns X-Request-ID and X-AnyRouter-Provider (the upstream that served it).
Routing preferences
provider constrains upstream selection without changing the model id:
{
"provider": {
"only": ["OpenAI", "Groq"],
"order": ["Groq", "OpenAI"],
"sort": "ttft",
"allow_fallbacks": true,
"max_price": {
"prompt": "1.00",
"completion": "2.00"
}
}
}AnyRouter tracks live TTFT and TPS per upstream and re-ranks on every request.
provider.sort | Sorts by | Direction | When to use |
|---|---|---|---|
cost | Listed input price per million tokens | Lowest price first | High-volume, cost-sensitive work |
ttft | Median time-to-first-token (ms) | Lowest latency first | Latency-sensitive workloads |
tps | Median tokens-per-second throughput | Highest first | Long-output generation |
Shortcut suffixes on the model id pick the sort for you: model:floor → cost, model:fast → ttft, model:nitro → tps. price, latency and throughput are aliases for cost, ttft and tps. Supported fields: only, ignore, order, sort, allow_fallbacks, max_price.prompt, max_price.completion, preferred_max_latency, preferred_min_throughput; others are accepted but ignored.
Response
{
"id": "chatcmpl-8yWq4JqLfEjJ9L",
"object": "chat.completion",
"created": 1760000000,
"model": "anthropic/claude-sonnet-4-6",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "NAT maps many private IP addresses to one public IP so devices behind a router can share a single internet connection."
},
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 28,
"completion_tokens": 30,
"total_tokens": 58,
"cost": 0.000342
}
}finish_reason | Meaning |
|---|---|
stop | Model completed naturally or hit a stop sequence. |
length | Hit max_tokens. |
tool_calls | Model invoked one or more tools; caller must run them and resume. |
content_filter | Upstream safety filter triggered. |
error | Upstream provider error; see error on the response. |
Cost of a request
usage.cost is the USD charged to your credits, 0 when nothing was charged (a free model or your own provider key, BYOK).
| Field | Type | Description |
|---|---|---|
usage.cost | number | USD charged for this request. 0 for free models and BYOK. |
usage.cost_details.upstream_inference_cost | number | USD the serving provider reported for the call, when it reports one. Present only when known. |
Where it appears:
- Non-streaming: on every response that includes
usage. - Streaming: with
stream_options.include_usage: true, on the one usage chunk (an emptychoicesarray) that follows the finish chunk, next toanyrouter_metadata. Without it, no chunk carriesusage, as in OpenAI's API. The charge is thenanyrouter_metadata.cost.amounton the finish chunk, which is sent once the provider ends the stream.anyrouter_metadata.cost.amountalways equalsusage.cost, on streaming and non-streaming responses alike. Either way each choice sends a non-nullfinish_reasonexactly once, and nothing butdata: [DONE]follows the last chunk. /responses:usage.costis the charged amount on non-streaming responses and on the terminalresponse.completedevent of a stream. The input/output split inusage.cost_detailsis not returned, because it never added up to the charge for cached tokens or non-list prices./messagesreports the charged amount asusage.costtoo: on the response, or on the finalmessage_deltaof a stream (Messages).
It matches your usage history. Once a chunk shows usage.cost, that amount is charged even if the client or provider disconnects before the stream ends.
Examples
curl https://anyrouter.dev/api/v1/chat/completions \
-H "Authorization: Bearer sk-ar-your-key" \
-H "Content-Type: application/json" \
-d '{
"model": "anthropic/claude-sonnet-4-6",
"messages": [
{ "role": "system", "content": "You are a concise assistant." },
{ "role": "user", "content": "Explain NAT in one sentence." }
],
"max_tokens": 200
}'from openai import OpenAI
client = OpenAI(
base_url="https://anyrouter.dev/api/v1",
api_key="sk-ar-your-key",
)
response = client.chat.completions.create(
model="anthropic/claude-sonnet-4-6",
messages=[
{"role": "system", "content": "You are a concise assistant."},
{"role": "user", "content": "Explain NAT in one sentence."},
],
max_tokens=200,
)
print(response.choices[0].message.content)import OpenAI from "openai"
const openai = new OpenAI({
baseURL: "https://anyrouter.dev/api/v1",
apiKey: "sk-ar-your-key",
})
const completion = await openai.chat.completions.create({
model: "anthropic/claude-sonnet-4-6",
messages: [
{ role: "system", content: "You are a concise assistant." },
{ role: "user", content: "Explain NAT in one sentence." },
],
max_tokens: 200,
})
console.log(completion.choices[0].message.content)Tool use
Pass a tools array with function schemas, then handle any returned tool_calls:
curl https://anyrouter.dev/api/v1/chat/completions \
-H "Authorization: Bearer sk-ar-your-key" \
-H "Content-Type: application/json" \
-d '{
"model": "openai/gpt-5.4-mini",
"messages": [
{ "role": "user", "content": "What is the weather in Tokyo?" }
],
"tools": [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get the current weather for a city",
"parameters": {
"type": "object",
"properties": { "city": { "type": "string" } },
"required": ["city"]
}
}
}
]
}'const response = await client.chat.completions.create({
model: "openai/gpt-5.4-mini",
messages: [{ role: "user", content: "What is the weather in Tokyo?" }],
tools: [
{
type: "function",
function: {
name: "get_weather",
description: "Get the current weather for a city",
parameters: {
type: "object",
properties: { city: { type: "string" } },
required: ["city"],
},
},
},
],
})See Tool use.
Errors
| Status | Meaning | Fix |
|---|---|---|
| 400 | Malformed body or invalid parameters. | Check the request against the field table above. |
| 401 | Missing or invalid API key. | Send a valid sk-ar-… key as Authorization: Bearer. |
| 402 | Out of credits or upstream BYOK billing failure. | Top up credits, or read metadata.upstream_message. |
| 404 | Unknown model. | Verify the id with List models. |
| 429 | Rate limit hit. | Back off; inspect X-RateLimit-* response headers. |
| 502 | Upstream 5xx or every fallback failed. | Retry; see metadata.upstream_attempts. |
See Errors for the full status table and retry recipes.
AnyRouter routes requests to third-party models. Models and outputs are provided as is, with no warranty; you use them at your own risk. Disclaimer