The loop is the whole idea
An agent repeats four steps. Take the conversation, ask a model, if the model asks for a tool then run it, then append the result and ask again. When the model answers without a tool call, you are done.
Everything else is detail. The model does not need a special agent endpoint, it needs a chat endpoint that accepts tools, which is the ordinary OpenAI Chat Completions shape.
The data shape
A conversation is a list of messages. A tool is a name, a description and a JSON schema. The model either returns content or a list of tool calls:
type Message = { role: "system" | "user" | "assistant" | "tool"; content: string; tool_call_id?: string }
type Tool = {
type: "function"
function: { name: string; description: string; parameters: object }
}
type ToolCall = { id: string; function: { name: string; arguments: string } }The loop keeps one message list and appends to it, so the model always sees the full history including its own tool results.
A minimal loop
This runs until the model stops asking for tools, with a hard cap so a bug cannot spin forever:
const messages: Message[] = [
{ role: "system", content: "You are a helpful assistant with tools." },
{ role: "user", content: input },
]
for (let step = 0; step < 8; step++) {
const reply = await call({ model: "anyrouter/auto", messages, tools })
messages.push({ role: "assistant", content: reply.content ?? "" })
if (reply.toolCalls.length === 0) {
console.log(reply.content)
break
}
for (const call_ of reply.toolCalls) {
const result = await runTool(call_.function.name, JSON.parse(call_.function.arguments))
messages.push({
role: "tool",
tool_call_id: call_.id,
content: JSON.stringify(result),
})
}
}Bound the tools
Give the model only the tools it needs, and validate every argument before running anything. A tool that writes to your system should require an explicit confirmation step, not a model decision.
Keep the tool list small. A model choosing between three tools is far more reliable than one choosing between thirty, and each tool description is a prompt the model reads on every turn.
Why the gateway helps an agent
An agent makes many calls, so a rate limit or an outage shows up fast. AnyRouter retries a failed upstream and can fail over to another model for the same route, so the loop keeps running.
Pin a stronger model for the planning turns and a cheaper one for the tool-selection turns. Free models are $0 per token on Go and up, with no daily cap, which makes an experimental loop cheap to run.
Pick a plan and start
Go is $2 a month for $4 in credits.
Pro is $10 a month for $20 in credits.
Pro+ is $45 a month for $100 in credits.
Max is $100 a month for $300 in credits.
Create an account, pay on the next screen, then call the model.