Latency Overhead
How much latency AnyRouter adds compared to calling the upstream provider directly, and when it is faster.
Through AnyRouter a request adds a small, predictable delay, and can be faster than going direct if you are far from the provider.
Path: your app → AnyRouter Worker (CF edge) → Cloudflare AI Gateway → provider. Responses stream back without buffering; billing and audit writes run via waitUntil, off the critical path.
| Stage | Typical wall-clock |
|---|---|
| CF Worker dispatch | ~0 ms warm, 5–20 ms cold |
| Auth + workspace lookup (D1 read, KV cache) | 1–10 ms |
| Routing + cooldown check | 1–5 ms |
| Body build / translate | <1 ms |
| AI Gateway hop | 10–30 ms |
| Streaming pipe-through | 0 ms per byte |
| Billing + audit writes | 0 ms on the critical path |
Net steady-state add: about 15–50 ms TTFB, 10–35 ms TTFT.
Faster than direct when
- You are far from the provider origin: TLS ends at the nearest CF edge and a warm pool covers the long haul.
- Your serverless backend cold-starts; AnyRouter's Worker is almost always warm.
- The provider rate-limits or has an outage:
body.modelsmoves to the next candidate faster than a retry loop (Model Fallbacks). - You want multi-provider redundancy without code changes (provider routing).
Slower when
- You already run in the provider's region (one fewer hop direct).
- The first request hits a cold Worker isolate (rare).
- You have a strict TTFB budget on small non-streaming responses where 20 ms matters more than reliability.
Measure it
Run each at least a few times to warm caches; the first request is not representative.
# Through AnyRouter
curl -w 'ttfb=%{time_starttransfer}s total=%{time_total}s\n' -o /dev/null -s \
-X POST https://anyrouter.dev/api/v1/chat/completions \
-H "Authorization: Bearer <ANYROUTER_API_KEY>" \
-H "Content-Type: application/json" \
-d '{"model":"openai/gpt-5.4-mini","messages":[{"role":"user","content":"hi"}],"max_completion_tokens":10}'
# Directly to the provider (your own key)
curl -w 'ttfb=%{time_starttransfer}s total=%{time_total}s\n' -o /dev/null -s \
-X POST https://api.openai.com/v1/chat/completions \
-H "Authorization: Bearer <OPENAI_API_KEY>" \
-H "Content-Type: application/json" \
-d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"hi"}],"max_tokens":10}'AnyRouter routes requests to third-party models. Models and outputs are provided as is, with no warranty; you use them at your own risk. Disclaimer