AnyRouter Docs

A/B Compare Models in the Playground

Send the same prompt to several models with one key and compare answers, latency, and cost.

Compare candidate models on the same prompt by changing only the model field. Each response reports latency and token usage, and every run shows in your usage explorer.

In the Playground

  1. Open the Playground. Your account key is attached.
  2. Pick a model and send a prompt.
  3. Switch the model and send the same prompt. Compare answers, latency, and tokens in the metrics footer.

In code

import { AnyRouter } from "@anyr/sdk"

const client = new AnyRouter() // reads ANYROUTER_API_KEY

const prompt = "Rewrite this commit message to be clear and concise: 'fix stuff'"

for (const model of ["openai/gpt-5.4-mini", "anthropic/claude-haiku-4-5", "z-ai/glm-4.7-flash"]) {
  const started = Date.now()
  const res = await client.chat.completions.create({
    model,
    messages: [{ role: "user", content: prompt }],
  })
  console.log(`\n=== ${model} (${Date.now() - started}ms) ===`)
  console.log(res.choices[0].message.content)
  console.log("tokens:", res.usage?.total_tokens)
}

Add "session_id": "model-bakeoff-2025-01" to the request body to group the runs in your logs.

AnyRouter routes requests to third-party models. Models and outputs are provided as is, with no warranty; you use them at your own risk. Disclaimer