Memory costs are climbing sharply, and Cloudflare runs massive distributed storage systems, including its CDN, that depend on efficient memory use. To stretch existing hardware further, the team prototyped Cache Transcoding: encoding eligible assets with Zstandard (zstd) inside Pingora before writing them to disk. The tradeoff is a minor CPU bump for significant storage and cross-data center bandwidth savings.
The Core Idea: Compress Once, Benefit Many Times
When an eligible response enters the cache, the proxy encodes it with zstd, stores it in that compressed form, and only decodes it right before serving the client. This means the compression cost is paid once per asset, while the storage and bandwidth savings accumulate on every subsequent reuse. In initial testing, eligible assets shrank to about one-third of their original on-disk size on average. The estimated extra CPU cost in the origin-facing proxy was small, but that’s the deliberate trade: a few percent more CPU for petabytes of effective cache capacity and reduced inter-data center transfers.
Zstd, a lossless compression algorithm open-sourced by Facebook in 2016, balances compression ratio with speed. In Cloudflare’s earlier browser compression testing, it compressed data 42% faster than Brotli with nearly the same file size, and produced files 11.3% smaller than gzip at comparable speed. The prototype uses zstd level 3, capturing most of the benefit without turning cache fills into a CPU bottleneck.
Not Everything Is Worth Compressing
Transcoding isn’t about compressing everything. Images, video, and fonts are already compressed; in Cloudflare’s traffic sample, this media slice represented 21.4% of requests but 63.3% of bytes. Recompressing them would waste CPU. Compressible text—HTML, JSON, CSS, JavaScript—represented 67.3% of requests and 22.3% of bytes. Within that text slice, about 71% arrived uncompressed and compressed well, achieving roughly 2.8x compression in the controlled test corpus.
The eligibility checks are strict: the prototype only transcodes 200 OK responses when Content-Encoding is unset, Content-Type is compressible text, and Content-Length is at least 4 KiB. This threshold removed many tiny requests while excluding only about 1% of eligible bytes. The team initially considered limiting transcoding to popular content, but that didn’t help—decoding happens on every serve, so restricting to hot assets reduced storage savings without cutting CPU proportionally. The simpler policy—transcoding all eligible compressible text at or above 4 KiB—captured nearly all the storage benefit within the CPU budget.
How Cache Transcoding Works in Pingora
On a cache miss, the Pingora-based proxy encodes the body with zstd before writing to disk. The cache metadata records that the stored representation is compressed and preserves the original content length. On a cache hit, the stored zstd object is read and decoded. With Tiered Cache, the compressed representation moves between tiers, and decoding only happens on the client-facing hop.
If the lower tier misses but the upper tier has the object, the compressed object transfers directly between tiers—no origin involvement, no re-encoding. A storage encoding marker prevents an object from being encoded more than once; a cache layer receiving an object can see it’s already zstd and preserve it.
Testing and Next Steps
The team exercised the prototype against a controlled test zone, correlating requests across logs, metrics, and traces. One performance campaign sent over a million requests across 10 cache servers, half with Tiered Cache disabled and half enabled. The two test assets, roughly 195 KiB and 272 KiB, compressed by about 2.8x—deliberately compressible to validate the architecture, not to represent the entire internet.
The results show the trade is favorable under tested conditions: content preserved, CPU within budget. Next steps include evaluating higher zstd levels, testing broader content types and sizes, and exploring range requests, pre-compressed origins, and passing compressed objects directly to downstream components that support it. The 4 KiB threshold and zstd level are parameters, not permanent limits—starting conservative allowed them to measure the architecture’s behavior before tuning.
For product builders, the lesson is clear: sometimes the cheapest optimization is changing how you store what you already have. Cache Transcoding doesn’t require new hardware—it makes existing hardware hold more by representing assets more efficiently. The tradeoff between CPU and storage is worth examining in any system where reads vastly outnumber writes.
Sources
AI-assisted summary compiled from the sources above, reviewed by a human before publishing.