Summary: Cloudflare shipped Transformers, a SQL-based transformation step inside Logpush that filters, redacts, and reshapes logs before export. It was announced on October 12, 2026, alongside Logpush availability on all self-serve plans, and works on Free through Enterprise tiers. For builders, this removes the sidecar service most teams otherwise run just to clean logs before they hit a SIEM or object storage — and cuts the volume (and cost) of data they pay to ingest and store.
The problem it solves
Most observability teams piping Cloudflare logs downstream need to do the same three things first: drop health-check traffic, remove or mask sensitive fields, and reshape records to match their own schema. Until now that meant deploying, scaling, and monitoring a separate processing service between Logpush and the destination — real infrastructure, real maintenance, real failure modes.
A Transformer is just a SQL query scoped to a Logpush dataset. Records failing the WHERE clause are dropped; the SELECT output becomes the exported record. You can rename, nest, or remove fields, redact part of a client IP, and add static fields — for example, a provider tag to normalize logs pulled from multiple CDNs. Output stays NDJSON, and everything is managed through Transformer Studio in the dashboard or the API, with versioned history per change.
The engineering choice worth stealing
The interesting part for anyone building data pipelines is what Cloudflare decided not to do. Their first prototype ran customer WebAssembly in Workers, but that would have put a network call in the export hot path — a dependency that could time out and retry at exactly the moment throughput matters most.
So they kept execution in-process inside Bufstream, their Go delivery service, embedding Apache DataFusion, a Rust query engine on Arrow columnar memory. In their benchmark, a prepared transform against a synthetic 5,000-row batch took about 3 ms — roughly 1.6 million rows per second. The Go/Rust boundary is crossed via Arrow IPC over cgo, with explicit ownership: Rust never retains the input pointer, and Go copies and frees the output. Batches, not rows, cross the boundary, avoiding per-record parsing.
Two details matter operationally. Transforms fail closed — if a transform can’t execute, Bufstream re-attempts rather than shipping raw data that would violate your redaction assumptions. And persistently failing jobs get progressively delayed retries, then automatic disabling, so one broken job can’t starve healthy ones. A Transformer Health view surfaces success rates, data volume before and after, and duration.
What this changes for your stack
Three practical takeaways:
- Cost: you currently pay to export, ingest, and store records you discard on arrival. Filtering pre-export shrinks all three bills. If you’re already tracking downstream observability spend — the same instinct behind cutting token and retrieval costs in LLM pipelines — this is the log-pipeline version of that discipline. We’ve covered related Cloudflare observability work before, like one model call scoring audio and video alongside text.
- Compliance: redacting IPs and sensitive fields before delivery means raw data never lands in your SIEM, which simplifies data loss prevention and retention policies.
- Architecture: the pattern — embed execution in the existing pipeline, keep the boundary batch-oriented, fail closed — is a reusable template for anyone adding user-defined logic to a high-throughput path.
One caveat: the in-scope benchmark is synthetic, so real-world latency on your actual batch shapes and query complexity may differ. Next step is cheap either way — open Transformer Studio, preview a query against a sample record, and attach it to a matching job to see the before/after volume in the Health tab.
Sources
View the original post on X