# How we cut our D1 bill by moving request logs to Cloudflare Basin

> AnyRouter wrote about 100M rows a month to D1, almost all of it request logs. We stopped logging refusals nobody reads and moved the full log to Cloudflare Basin, which stores it as Iceberg tables. This post covers the numbers, the design, and the tradeoffs.

- Date: November 7, 2026
- Category: engineering
- HTML: https://anyrouter.dev/blog/d1-bill-request-logs-basin

## D1 cost was row writes

D1 bills per million rows written after an included monthly allowance. Every index on a table adds one billed row per insert, and row size does not matter. Storage is charged per GB-month after an included allowance.

On 7 November 2026 we measured 3.2 to 4.0M rows written per day. That is about 100M a month, about 2x the included monthly allowance. The peak was 8.7M in a day on 27 October, and earlier fixes had already halved it. The database file was 6.45 GB, so storage was not the problem.

The top writers over 24 hours were all request logs:

- `INSERT INTO generations`: 1.565M rows from 141k calls, about 11 rows each, because the table has 14 indexes.
- `INSERT INTO generation_attempts`: 583k rows from 117k calls, about 5 each.
- The hourly rollup `workspace_usage_hourly`: 213k rows from 85 calls.

## Most of it was refusals

Of the 110k `generations` rows written per day, 63k had status `rejected`. 54k of those came from one API key that swept 286 BYOK-only models every 4 minutes, with no BYOK key attached. Each refusal wrote about 16 D1 rows for a 4xx response that nobody reads.

We checked the indexes first. No unused index is left on `generations` or `generation_attempts`, because earlier migrations already dropped 14. Dropping indexes was not a lever.

## Fix 1: refusals write no D1 row

A request that is refused before any upstream is tried now writes no D1 row. That covers a BYOK-only model without a key, an unknown model, a key that is not allowed to use the model, and the billing and free-model gates.

The count is not lost. Analytics Engine still counts every refusal by error code, and the Logs page shows an "N requests refused" note. One case keeps its row on purpose: a platform-side empty dispatch, where the router fails a model it should serve. Route health reads those rows.

We checked this on production. The sweeper's rows went from about 620 per 10 minutes to 0.

## Fix 2: the full log goes to Basin

Refusals were the largest single source, but not the only one. Every successful request still wrote about 16 rows, and the log grows with traffic. So we moved the full log out of D1.

Cloudflare Basin is a data lake product. A Pipelines stream takes typed records from a Worker binding and writes them as Parquet files into an Iceberg table in an R2 bucket. You query the table with Basin SQL. We created three streams, each with a typed schema:

- `generations`: 66 columns, covering every D1 column.
- `generation_attempts`: 29 columns.
- `generation_bodies`: 12 columns, with the full request and response text, up to 1 MB per side. Requests under zero data retention are never stored.

Each stream sinks to its own table in one Basin Catalog bucket, as Parquet with zstd compression. A file rolls every 300 seconds, and automatic compaction merges files toward a 128 MB target.

The writes come from the existing telemetry Worker over a service binding, so they run on that Worker's own CPU budget and not on the gateway's request path. The gateway log sync sends captured bodies from the workflow Worker. A unit test fails if a D1 column is missing from the Basin schema, because Basin drops records that do not match.

```ts
// Pipelines binding declared in wrangler.jsonc
await env.BASIN_GENERATIONS.send([
  {
    id: row.id,
    model_id: row.model_id,
    status: row.status,
    cost_usd: row.cost_usd,
    created_at: "2026-11-07T10:15:00Z",
    // ...the rest of the schema's columns, null when missing
  },
])

// Basin SQL, run against the Iceberg table
const query = `
  SELECT model_id, COUNT(*), SUM(cost_usd)
  FROM anyrouter.generations_v1
  WHERE created_at > now() - INTERVAL '1 day'
  GROUP BY 1
  ORDER BY 2 DESC
  LIMIT 20
`
```

_One send per batch from the telemetry Worker, then the kind of query we run for cost by model. The real record carries all 66 columns of the schema._

## What it costs

Pipelines ingest is free, with 50 GB a month of transforms and sink included, then usage-based pricing per GB for transforms and for Parquet and Iceberg sinks. R2 storage is charged per GB-month with an included allowance. Basin Catalog includes 1M operations a month. Basin SQL is charged per TB scanned, with an included monthly allowance and a small minimum per query.

Our volume is about 260k log rows a day, which is roughly 6 to 12 GB a month uncompressed. That fits inside the included allowances at today's traffic, so the Basin path adds effectively no extra cost today. Here is how the two compare as traffic grows, in relative terms:

| Traffic | D1 per month | Basin per month |
| --- | --- | --- |
| 1x (today) | about 100M rows written | about 6–12 GB, inside included allowance |
| 10x | about 1B rows written | about 60–120 GB, 1 order of magnitude cheaper |
| 100x | about 10B rows written | about 0.6–1.2 TB, 1 order of magnitude cheaper |

D1 cost scales with rows written, and each index multiplies it. Basin cost scales with bytes stored and scanned.

## What we gave up

Basin SQL runs at analytics speed, which means seconds and not milliseconds. New rows appear after minutes, because of batching. Iceberg is append-only, so a row is written once at its final state, and deletes need care.

So D1 stays the hot store. The Logs page still reads D1, which is fast and updatable. Basin is the system of record for history, analytics and future datasets. It keeps everything with no deletion, and the dashboard only limits what it shows.

Three constraints are worth knowing before you copy this. A stream schema cannot change after creation, so a change means a new stream and a new table name such as `generations_v2`. A sink cannot target an existing Iceberg table. And records that do not match the schema are dropped silently, so we watch the Pipelines user error metrics after any change to the mapper.

## How the decision was made

anybot agent made this design decision. It analyzed the system, measured where the writes came from, compared the options on cost, and chose the cheapest one that kept the Logs page working. The owner reviewed the decision before it shipped.

Next, we plan to backfill the existing D1 history into Basin, then move free-request logs off D1, make the hourly rollup rewrite only the hours that changed, and fold our older R2 archives into Basin. Those archives are an NDJSON lakehouse, AI Gateway log shards, and a daily D1 copy job.
