Summary: On July 29, Andrey Mikhaylov — an engineer whose work centers on iOS and Metal — put TurboFieldfare on Show HN. To produce each token, the router first picks the top-8 expert IDs, and those weights are then pulled off SSD through bounded parallel pread calls into a 16-slot LFU cache, where later tokens can draw on them again. Written from scratch in Swift and Metal and released under Apache-2.0, the engine runs Gemma 4 26B-A4B on any Apple Silicon Mac inside roughly 2 GB of RAM, machines with no more than 8 GB of unified memory included.
On July 29, Andrey Mikhaylov — an engineer whose work centers on iOS and Metal — put TurboFieldfare on Show HN. Written from scratch in Swift and Metal and released under Apache-2.0, the engine runs Gemma 4 26B-A4B on any Apple Silicon Mac inside roughly 2 GB of RAM, machines with no more than 8 GB of unified memory included. The post drew 919 points and more than three hundred comments. The project takes its name from the fieldfare, a thrush the author likes best, and the README says outright that Google has nothing to do with it and that it is not a wrapper around MLX or llama.cpp.
How a 26B Model Squeezes Into 2 GB
The explanation sits in the mixture-of-experts layout of Gemma 4 26B-A4B: about 26 billion parameters in total, of which only some 3.88 billion are engaged for any single token. TurboFieldfare takes that opening without ceremony — the whole model never enters memory. Only the shared core (roughly 1.35 GB) and an FP16 KV cache stay resident. To produce each token, the router first picks the top-8 expert IDs, and those weights are then pulled off SSD through bounded parallel pread calls into a 16-slot LFU cache, where later tokens can draw on them again.
Precision is deliberately uneven across the model. Embeddings, attention and expert weights live as MLX affine 4-bit with group 64, whereas the router is held at 8-bit, on the reasoning that a misrouted token hurts more than a coarser weight does. Chunks of at most 128 tokens make up the prefill batch, which lets one expert fetch answer several rows at once. Layer type decides the KV cache layout too: circular storage across the 25 sliding-window layers, linear across the 5 full-attention layers.
Why this only works for a MoE deserves spelling out. Run the model dense, and all 26B parameters get touched on every token, leaving nothing that could be left on disk. The gap between 26B total and 3.88B active is what the streaming arithmetic rests on: no single token drags more than a few hundred megabytes of cold experts off the drive, and whatever locality real workloads display is caught by the LFU cache.
Benchmarks: 5 to 6 Tokens per Second on an 8GB Air
These are the figures the author measured: 5.1 to 6.3 tokens per second while decoding on an 8GB M2 MacBook Air, against 31 to 35 tokens per second on a 24GB M5 Pro. Storage is where the bill lands — the text model runs about 14.3 GB. During installation, byte ranges come streaming off a pinned Hugging Face revision and are repacked into a .gturbo layout on arrival, so a complete checkpoint never sits staged on disk. Text-only mode is fine on an M1, while the optional 1.1 GB vision pack demands M2 or newer; macOS 26 and Metal 4 round out the requirements.
Six Swift products arrive with the repository, reaching well past the engine — the core library, a native Mac app, a one-shot decode service, a CLI, and a loopback OpenAI-compatible Chat Completions server with function tools (no auth, no TLS, strictly local) — together with the streaming installer. Documented alongside them are 103 audited experiments, a count that probably reveals more about how the tuning went than any headline benchmark could.
Why This Line of Attack Merits Attention
RAM is the wall that other local runtimes assume cannot be moved: fail to fit the model and you cannot run it. Here SSD becomes part of the memory hierarchy, random-read latency is the currency spent to reclaim RAM, and MoE sparsity is what keeps that bargain livable. Nobody framed the issue better than the top-voted Hacker News comment, which asked why we insist on shoving the entire model into memory at all. The answers beneath it were every bit as blunt: knowing which knowledge a token needs is a research problem nobody has solved, and MoE covers just a share of it.
Two lessons land for anyone building a product. The first is that 8GB entry-level hardware is once again a workable target, which pushes the floor for on-device AI a notch lower. The second is a trade-off left in plain sight: you surrender disk space and random-read latency, and what you get back is memory. Deciding whether that is a good deal comes down to what your product is really after — running at all, or running quickly; an interactive assistant and a background summarizer will answer that differently.
One extension invites itself: aim the same expert-streaming scheme at bigger MoE checkpoints, where the core that stays resident would not grow but the disk footprint would. iPhone and iPad ports appear on the future-work list and chase the same idea — the payoff from this trade-off is largest exactly on devices where memory runs short.