On June 10, 2026, Google released DiffusionGemma, an experimental open model that brings text diffusion into the Gemma family for the first time. It is built on the Gemma 4 architecture, informed by the company’s Gemini Diffusion research, ships under Apache 2.0, and is live on Hugging Face as google/diffusiongemma-26B-A4B-it.
The pitch is a single number: speed. Google claims up to 4x faster text generation on dedicated GPUs versus autoregressive models — more than 1,000 tokens per second on a single NVIDIA H100, and over 700 on a consumer RTX 5090.
Why Diffusion Generates Faster
An autoregressive model emits one token at a time, which pins the bottleneck to memory bandwidth. DiffusionGemma generates onto a “canvas” instead: it starts with 256 random placeholder tokens, then runs iterative refinement passes — each pass locks in the tokens it is confident about and uses them as context for the next — until the output converges. Bidirectional attention lets every token see every other token, shifting the bottleneck from memory bandwidth to raw compute. The model card adds implementation detail: an autoregressive encoder handles prefill and keeps the KV cache, while the decoder applies bidirectional attention over the canvas; in low-batch, FP8 settings it exceeds 1,100 tokens per second on one H100.
The bidirectional setup also makes it good at non-linear work: in-line editing, code infilling, amino acid sequences, math graphs. A fine-tuned version by Unsloth can solve Sudoku — a task Google pointedly notes autoregressive models struggle with.
A 26B-A4B MoE That Fits a Consumer GPU
The model is a 26B-A4B mixture of experts: the model card lists roughly 25.2B total parameters with 3.8B active per forward pass, selecting 8 of 128 experts plus one shared expert. It is actually a multimodal image-text-to-text model, with a ~550M-parameter vision encoder, support for video up to 60 seconds, a 256K-token context window, and a 262K vocabulary; 35+ languages work out of the box (140+ appeared in pretraining). Quantized, it “fits comfortably” within the 18GB VRAM budget of high-end consumer GPUs. The encoder-decoder split is the practical detail: prefill behaves like any autoregressive model with a KV cache, so long prompts do not pay a diffusion penalty; only the generation phase runs on canvases.
The Quality Cost: Still Trails Gemma 4
Google does not hide the trade. Against the same-size Gemma 4 26B-A4B: MMMU Pro 54.3% vs 73.8%, AIME 2026 without tools 69.1% vs 88.3%, GPQA Diamond 73.2% vs 82.3%, and MMLU Pro 77.6% vs 82.6%. The one win is HLE without tools, 11.0% vs 8.7%. Google’s own guidance is to use standard Gemma 4 when quality matters most. The speed story also has limits: the numbers target local, low-concurrency inference. In high-QPS cloud serving, parallel decoding shows “diminishing returns and can result in higher serving costs,” and Apple Silicon Macs may not see the same speedup because of their lower compute-to-memory bandwidth ratio. For sampling, Google recommends Entropy-Bounded Denoising: at most 48 steps, with temperature decaying from 0.8 to 0.4. That tuning profile is a reminder that diffusion serving is operationally different — step counts and entropy bounds replace the familiar temperature-and-top-p dials.
The Ecosystem: Serving and Fine-Tuning
On the serving side, DiffusionGemma runs through MLX, vLLM (with Red Hat support), and Hugging Face Transformers, with llama.cpp support “arriving soon.” Fine-tuning routes include Hackable Diffusion (JAX), Unsloth, and NVIDIA NeMo. On the cloud side it is available in the Gemini Enterprise Agent Platform Model Garden and via NVIDIA NIM, with NVFP4 kernels for RTX 5090/4090, Hopper, and Blackwell — covering DGX Spark, DGX Station, and RTX PRO workstations.
What It Means for Developers
Three takeaways. First, this is not a daily-driver replacement — it is a fast path for specific jobs: local inference, in-line edits, code infilling, anything that benefits from whole-canvas context. Second, when llama.cpp support lands, it redefines what consumer hardware can do for text generation; an 18GB VRAM threshold puts it within reach of most gaming cards. Nor is this a signal that anyone’s default cloud serving flips to diffusion: the 4x number is a low-concurrency result, and the economics of multi-tenant GPU fleets run on a different equation. Third, by open-sourcing the diffusion route under Apache 2.0, Google is inviting the whole ecosystem to validate the architecture bet (see also: our 2026 opening outlook).
Sources
AI-assisted summary compiled from the sources above, reviewed by a human before publishing.