Latency Overhead

How much latency AnyRouter adds compared to calling the upstream provider directly, and when it is faster.

Through AnyRouter a request adds a small, predictable delay, and can be faster than going direct if you are far from the provider.

Path: your app → AnyRouter Worker (CF edge) → Cloudflare AI Gateway → provider. Responses stream back without buffering; billing and audit writes run via waitUntil, off the critical path.

StageTypical wall-clock
CF Worker dispatch~0 ms warm, 5–20 ms cold
Auth + workspace lookup (D1 read, KV cache)1–10 ms
Routing + cooldown check1–5 ms
Body build / translate<1 ms
AI Gateway hop10–30 ms
Streaming pipe-through0 ms per byte
Billing + audit writes0 ms on the critical path

Net steady-state add: about 15–50 ms TTFB, 10–35 ms TTFT.

Faster than direct when

  • You are far from the provider origin: TLS ends at the nearest CF edge and a warm pool covers the long haul.
  • Your serverless backend cold-starts; AnyRouter's Worker is almost always warm.
  • The provider rate-limits or has an outage: body.models moves to the next candidate faster than a retry loop (Model Fallbacks).
  • You want multi-provider redundancy without code changes (provider routing).

Slower when

  • You already run in the provider's region (one fewer hop direct).
  • The first request hits a cold Worker isolate (rare).
  • You have a strict TTFB budget on small non-streaming responses where 20 ms matters more than reliability.

Measure it

Run each at least a few times to warm caches; the first request is not representative.

# Through AnyRouter
curl -w 'ttfb=%{time_starttransfer}s total=%{time_total}s\n' -o /dev/null -s \
  -X POST https://anyrouter.dev/api/v1/chat/completions \
  -H "Authorization: Bearer <ANYROUTER_API_KEY>" \
  -H "Content-Type: application/json" \
  -d '{"model":"openai/gpt-5.4-mini","messages":[{"role":"user","content":"hi"}],"max_completion_tokens":10}'

# Directly to the provider (your own key)
curl -w 'ttfb=%{time_starttransfer}s total=%{time_total}s\n' -o /dev/null -s \
  -X POST https://api.openai.com/v1/chat/completions \
  -H "Authorization: Bearer <OPENAI_API_KEY>" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"hi"}],"max_tokens":10}'

AnyRouter routes requests to third-party models. Models and outputs are provided as is, with no warranty; you use them at your own risk. Disclaimer