How our CI/CD changed when coding agents took over

Most changes to AnyRouter are now written, checked and shipped by coding agents. They push and deploy many times a day. When writing code got fast, the pipeline around it became the thing everyone waited on.

We ran GitHub Actions CI for five months. Then we changed our CI/CD design four times in one month, between 4 October and 3 November. Each change fixed one bottleneck and showed us the next one. This post walks through the stages in order, with the dates and the commits or PRs that made each change.

All numbers here were measured on 3 November 2026 from git history, GitHub, our deploy log and a lead-time script. Where a number has a gap or a caveat, we say so next to it.

On 23 May 2026 we set up the usual pipeline (#321). Each pull request ran CI on GitHub Actions and got a preview deploy. A merge to main deployed from GitHub Actions.

This works well for a team of people. It does not work well for a fleet of agents. An agent pushes, waits for CI, fixes what failed and pushes again. Every one of those pushes runs the full pipeline. We did not measure the queue at the time, but the waiting was easy to see.

It also costs money. Our repository is private. GitHub documents that Actions on private repositories include a monthly amount of usage, and that you pay per minute after that. Standard hosted runners are free only for public repositories. Agents push all day, so we expected to pay for most of those minutes. We did not measure that cost.

On 4 October we turned GitHub Actions CI off (#4452). Only the release workflows still run there. The same checks now run as pnpm local-ci on one dedicated machine, and that run signs a commit status on the PR. Agents develop and deploy from that machine too.

One machine brings its own limits. It has 15 GB of memory, and many agents share it. Three changes made it work.

• Per-Worker deploys (10 October). AnyRouter runs as 19 Cloudflare Workers. A deploy now ships only the Workers whose code changed since their own last deploy. Each Worker version is stamped with the commit it was built from, so the next deploy knows where to diff from.
• A heavy lock (12 October, #5026). Installs, builds, typechecks and test runs take one shared lock, scoped to their heavy phases. Before that, parallel builds ran the machine out of memory and the out-of-memory handler killed agent sessions.
• One merge script (14 October, #5492). merge-pr.sh became the single way a lane PR lands, with the same checks every time.

We wrote about the per-Worker deploys and about making local CI faster in earlier posts. This post is about the flow around them.

Local CI took away the cost. It did not take away the wait. So on 28 October we changed the merge gate (#6840). A PR merged on one independent review, a review:approved label plus a signed status, with no per-PR CI run. Tests ran once on main after a batch of merges, and failures were fixed forward. A PR with no review after 2 hours was flagged as stale.

Then the volume caught up with us. In the week of 26 October, 821 PRs were merged. Each one got its own reviewer agent, its own worktree and its own dependency install. A reviewer that sees one small PR cannot see how it interacts with the PR next to it. Per-PR review at that volume did not make sense.

The lead-time numbers show the cost. In that week, the median PR went from opened to live in 1.4 hours, but the slowest 10 percent took 22.2 hours or more. Most of that tail came before the merge, not between the merge and the deploy. The lead-time chart is in the metrics section below.

On 3 November the manager session was running 8 lanes at once. They queued behind each other's test runs on the heavy lock for 15 minutes or more. Two lanes edited the same files and had to wait on each other's PRs. So we changed the rule (#7798).

• At most 2 implementation lanes at once. Everything else waits in a queue.
• Lanes are fixed by code area, such as catalog and pricing, or model pages UI. A new request joins the lane that owns its area.
• A lane stacks its queued changes into one larger change with one review.

This idea came partly from Anthropic's article Building agents: when to fold subagents into your main agent loop. Its advice is to keep a subagent only for parallel work in a fresh context, for measured context bloat, or when one agent struggles with its tools or instructions. Otherwise, fold it into the main loop. A handoff between agents loses detail, and every extra agent adds latency. We saw both.

Fat PRs still had a round trip. On 3 November a one-line test fix still needed a worktree, an install, a PR, a reviewer and a merge before the deploy it unblocked. So we dropped PRs (78675034b).

Now an agent works on main, or in a lane worktree that pushes to main. It reviews and checks locally, commits each working step as a checkpoint, rebases, and pushes. The push is the backup and the shared state between machines. Deploys run from local main.

Review did not go away. After a range lands, a reviewer reads git diff <last-reviewed>..<head> and lists issues. For small changes that reviewer is the main session. Billing, pricing, auth, secrets, migrations and routing still get an independent reviewer before the push. Every fix lands as a new commit. We never amend or force-push a reviewed commit, so the review trail stays readable.

Release-please PRs are the one kind of PR left. The owner merges those by hand.

Measuring each stage turned up problems that no check reported. Every check passed while each of these cost us time or let a bug through.

• A deploy loop that fed itself. Stray untracked files in the deploy checkout made every deploy stamp its Worker versions as +dirty (the manager's diagnosis counted 5 such files). A deploy cannot diff from a dirty stamp, so the next deploy rebuilt the whole fleet, and stamped +dirty again because the files were still there. All 51 deploys since 1 November were full-fleet deploys. Stray untracked files no longer mark a deploy dirty (2ee30c20f).
• A cache that never hit. The deploy skips its typecheck when the same commit already passed. That cache treated the stray untracked files as a dirty tree, so it hit 0 of 51 times. Typecheck averaged 197 seconds per deploy. It was the same root cause as the loop above. The cache now ignores untracked files that TypeScript never reads (fbe45259b).
• A smoke run too soon after a deploy. On 3 November (measured), the first production smoke run right after a deploy failed 52 of 115 checks with 503 database_unavailable. A rerun a few minutes later passed all 72 checks it did not skip. The failures came from the fresh deploy, not from a bug. We now wait before the post-deploy smoke.
• Vitest does not check types. Vitest strips types and runs the code. A lane's check ran only Vitest, and a test with a type error reached main. It surfaced at deploy, which stops at the first type error. Now every lane runs the root pnpm typecheck before it pushes and reads its real exit code.
• A fast-forward-only guard on main. On 29 October a lane's worktree was removed while the lane still used it. Its next command chained a cd that failed, so a git push --force-with-lease ran from a stale checkout of main. main moved back for about a minute (#7089). The lane restored it and no commits were lost. When we allowed direct pushes to main, we kept that lesson. The pre-push hook allows fast-forward pushes and refuses any push that rewrites or deletes main (62d85ea2b).
• The commit hash is the identity. Every Worker version on Cloudflare carries anyr-sha=<commit>. That one field drives the per-Worker deploy diff, the rollback check that stops a stale checkout from rolling production back, and the lead-time numbers in this post. A metric keyed on a PR number breaks the day you stop opening PRs. A metric keyed on the commit does not.
• Deploy speed, in progress. A deploy took a median of 602 seconds over those 51 runs. The two fixes above landed on 3 November, so per-Worker deploys and the typecheck cache can work again. We are also working on a build cache that survives between deploys, and we are evaluating tsgo, the Go-based TypeScript compiler. We have not measured a result from any of these yet.

This is the current contract, adapted from the Main-only flow and Few lanes sections of our AGENTS.md. You can copy it and change the commands for your own repository.

# How we land changes (main-only flow)
- No pull requests. Work on main, or in a lane worktree that pushes to main.
- Keep each commit small and on one topic. Stage exact paths. Never git add -A.
- Check locally before every commit:
  - read the whole diff
  - targeted vitest: scripts/heavy.sh light -- npx vitest run <files> --maxWorkers=2
  - biome on the changed files
  - root pnpm typecheck for TypeScript changes; read its real exit code
    (vitest does not type-check)
- Billing, pricing, auth, secrets, migrations, routing: one independent
  reviewer reads the diff before you push.
- git commit, git pull --rebase origin main, git push origin main.
  Never force-push main.
- A review of the pushed range lands its fixes as new commits.
  Never amend, rebase or force-push reviewed commits.

# Few lanes
- At most 2 implementation lanes at once. Queue the rest.
- Lanes are fixed by code area. A new request joins the lane that owns
  its area. It does not get a new worktree.
- Fold before you fan out. Spawn a subagent only for parallel
  fresh-context work, a black-box review, or measured context bloat.
- One heavy command at a time per machine (install, vitest, tsc, build).

These charts use git history, GitHub and our deploy log. The deploy log covers only one machine, and some weeks are missing from it. Those weeks are left out of the deploy charts, not drawn as zero. So the deploy counts are a floor. The week of 2 November is partial. It runs to 3 November.

In August, the counts suggest many commits on main did not come from a merged PR. From late September, almost every commit was a squash-merged PR, and both lines peaked in the week of 26 October with 810 commits and 821 PRs. After the switch to main-only, the PR line should fall while commits continue.

Deploy time rose in October. Deploys became full builds of a growing fleet of 19 Workers, plus a typecheck gate. The dirty stamp above made every deploy since 1 November a full one.

Deploying the spoke Workers and the typecheck make up about half of the summed step averages. Those are the two things the dirty stamp breaks. A clean stamp lets a deploy skip unchanged Workers and skip a typecheck that already passed.

Our script scripts/delivery-metrics.ts measures lead time per commit. A commit starts when its PR opens, or when it is authored if it had no PR. It is live when a Worker version stamped with that commit, or a later one, is deployed. This table splits the PR flow into its two halves.

In the week of 26 October, the p90 tail sat almost entirely before the merge, at 21.7 hours. The wait was before the merge, not between the merge and the deploy. The partial week of 2 November is much faster, but it covers only 83 commits from the last days of the PR flow. It is not a result of the main-only flow.

The main-only flow is one day old. When we measured, it had 6 commits on main and none of them had been deployed yet. We have no after numbers for it, so we do not claim a speedup.

The earlier main rows are a handful of direct commits made during the PR era. They are too few to compare with anything.

We will rerun npx tsx scripts/delivery-metrics.ts --since 2026-10-12 after a few days of the new flow and the two deploy fixes. We will update this post with those numbers, including the ones that do not look good.