AGENTIC COMMONSAI industry briefings

繁中EN

TopicAI Chips & ComputePublished

All English articles

Cloudflare Cut Its AI Review Bill 68%: What to Steal

Summary: Cloudflare’s internal AI code reviewer crept from $1.19 to roughly $2.43 per review between spring and August 2026, and a mid-September optimization push brought it down to $0.77 — a 68% cut — while also dropping median review time from 4m 09s to 1m 32s. About 40% of the savings came from code changes, the rest from model and effort-level choices. The breakdown is worth reading closely because it maps directly onto cost problems most teams building agentic systems will hit.

The bill doubled for a boring reason: more tokens, better models

The cost climb wasn’t architectural drift. Cloudflare’s postmortem on the reviewer attributes most of the increase to model changes. Moving every reviewer onto Anthropic models in early August — a choice justified by better eval results on this workload — raised cost per job from $1.38 to $2.03. Meanwhile, tokens per review more than tripled from 0.91M to 2.99M. Their prompt-cache hit rate actually improved, from 85.7% to 97%, but a high hit rate on three times the tokens still costs far more.

When they finally audited spend, 77% of it went to cache writes and reads. The model was paying repeatedly for context it had mostly already seen. This is the same prefix discipline problem behind prompt caching economics: where your cache breakpoints sit determines whether shared context ever gets reused.

The prefix bug that cost the most

Every session opened with 11k–21k identical tokens — the same tool definitions and agent prompt across every repository. Yet the first request only found a cached prefix 14% to 26% of the time. The culprit: OpenCode v1 appended repository-specific environment details (working directory, date) into the same block that held the shared agent prompt, with a single breakpoint at the end. The shared prefix was never cached on its own, so one repo’s review could never warm the cache for another’s.

A plugin hook that split the block at a marker — and a strict rule that nothing repo-specific may appear before it — fixed this. Together with inlining a standards reviewer’s reference index into the prompt (eliminating three tool-call turns per session), the cost per review fell from $2.47 to $2.02 almost immediately.

Small fixes, compounding savings

The next tranche of changes went out September 21–23 and shaved about $0.20 per review. None was glamorous:

  • Removing tools sub-reviewers couldn’t use, which eliminated a wasted recovery call in 57%–76% of sessions
  • Reading shared context and patches in parallel rather than one file per turn
  • Skipping reviewers on re-reviews when nothing new changed (about two-thirds of jobs are re-reviews)
  • Stopping the loop after the review posts, instead of one more turn to write a summary nobody read
  • Detecting cancelled jobs and actually stopping them

One subtle trick: three reviewers now share an identical first request, with one designated to write the cache and the other two waiting up to 20 seconds to read it. It saves roughly nine cents per reviewer at a small latency cost.

Models and effort mattered more than code

Splitting the timeline at each release, code changes account for $0.65 of the $1.67 saved; model and effort changes account for $1.06. Opus 5.5 at medium effort showed no measurable quality drop versus Opus 5 at higher settings, and Sonnet 5.5 matched Sonnet 5’s findings at 47%–62% lower cost per run despite identical token prices — purely from using fewer tokens.

The interesting counterpoint: on the code quality reviewer, high effort cost 61% more but surfaced 58% more critical and warning findings, so they kept it there. The lesson isn’t “always lower effort” — it’s measure findings per dollar per tier, then tune each reviewer independently.

The honest caveat Cloudflare includes: their before/after comparisons aren’t controlled experiments. If you copy this playbook, instrument token usage and cache hits per first-request before you start, and A/B your quality metrics as you cut — the savings are real, but only measurement tells you what you traded away.

Sources

AGENTIC COMMONSOperated by PHLEGON LABS
ShareXEmail
Support us

Related reading

  1. Prompt Caching: The Prefix Discipline That Cuts Agent Bills

    A builder's guide to provider cache pricing, break-even math, and the configuration mistakes that push hit rates from 77% down to 24%.

    Cache

  2. Cloudflare's Vary Support: Decide What Variation Your Cache Can Afford

    Cloudflare now lets Cache Rules normalize, pass through, or bypass each Vary header, so caching stops fragmenting.

    Cloudflare

  3. Model Caching on HyperPod: What Changes When Weights Live on the Node

    HyperPod model caching pre-loads weights and images to local NVMe, cutting scale-out cold starts from tens of minutes to seconds.

    AWS