# Run one eval across many models

> Loop the same prompts over a list of catalog ids with one key and one client. Compare answers, latency, and cost side by side.

Loop the same prompts over many models

**Slug**: `model-evals`  
**URL**: https://anyrouter.dev/use-cases/model-evals  
**Markdown**: https://anyrouter.dev/use-cases/model-evals.md  
**Short link**: https://anyrouter.dev/u/model-evals  

## Why use this

Evaluating models from different vendors means one SDK and one key per provider, and cost lands on separate invoices you have to add up.

## Set it up

1. Run anyr models (or open /models) and pick the ids you want to compare.
2. Send the same prompt to each id through one AI SDK provider. Models on your own provider keys (OpenAI, Anthropic) work the same way.
3. Compare outputs and token usage, then check cost per request in Dashboard → Logs.

## Code

### Eval loop

```ts
import { createAnyRouter } from "@anyr/ai-sdk-provider"
import { generateText } from "ai"

const anyrouter = createAnyRouter()
const models = ["z-ai/glm-4.7-flash", "deepseek/deepseek-v4-flash", "google/gemma-4-31b"]

for (const id of models) {
  const started = Date.now()
  const { text, usage } = await generateText({
    model: anyrouter(id),
    prompt: "Summarize the CAP theorem in two sentences.",
  })
  console.log(id, Date.now() - started, "ms", usage.totalTokens, "tokens\n", text)
}
```

### anyr CLI

```bash
anyr models
```

## Related

- [ai-sdk](https://anyrouter.dev/use-cases/ai-sdk)
- [cheapest-llm-api](https://anyrouter.dev/use-cases/cheapest-llm-api)
- [unify-api](https://anyrouter.dev/use-cases/unify-api)
