In the first week of May 2026, ds4 — DwarfStar — an open-source project from Salvatore Sanfilippo, the creator of Redis, climbed to the Hacker News front page with roughly 500 points and about 160 comments. It is a local LLM inference engine written in C, with a plainly stated goal: to be “the best way to run a few excellent large language models on consumer hardware.” In a 2026 dominated by hosted APIs, antirez’s motivation is blunt — “AI is too critical to be just a provided service.”
What ds4 Is
ds4 ships as more than an engine. The repository provides the ds4 CLI, a native coding agent (ds4-agent), and an HTTP server (ds4-server) that listens on 127.0.0.1:8000 by default, plus eval, bench, and GGUF tooling. The supported model list is deliberately short: DeepSeek V4 Flash (with experimental vision), DeepSeek V4 PRO, and GLM 5.2, 5.3, and 5.3 Flash — and models can be dropped whenever a better replacement arrives. The recommended first run is the Q2 quantization. The project is MIT licensed; the core is C, the Metal backend is Objective-C, and CUDA has its own kernels. The license file retains GGML’s copyright notice — antirez states openly that some quantization layouts and CPU kernels were adapted from llama.cpp, and credits Georgi Gerganov.
Not a Generic GGUF Runner
ds4 deliberately diverges from general-purpose runners like llama.cpp and Ollama: it does not accept arbitrary GGUF files and only runs model files the project produces itself. The trade buys deep optimization for the DeepSeek and GLM architectures — aggressive routed-expert quantization and compressed KV caches. The deployment story also reaches beyond one machine. Metal is the primary target, aimed at Macs with 96 GB of RAM or more, with SSD streaming letting smaller machines run models larger than their memory. The CUDA port targets the DGX Spark above all and supports multi-GPU systems like the L40S; the ROCm port aims at Strix Halo machines. There is even RDMA-based tensor parallelism across two 128 GB Macs, and pipeline parallelism to pool RAM across machines.
Performance Numbers and the Hardware Map
The README offers concrete figures: a system with eight L40S cards reaches roughly 126 tokens per second of aggregate generation while serving 16 sessions, and the baseline sweep ran on an M5 Max with 128 GB of RAM. The quantization recipe is the soul of the project — antirez explains on his blog that DeepSeek V4 Flash, a quasi-frontier open-weights model, becomes practical with an extremely asymmetric 2/8-bit recipe, at which point 96 or 128 GB of RAM is enough. Speculative decoding is optional (MTP for GLM, DSpark for Flash), and there is directional steering that works without rebuilding the KV cache. The project is labeled beta and described as changing very fast.
Software Built in a Week, the AI Way
antirez says the core of ds4 took one week to build, during which he worked 14 hours a day — far above the 4 to 6 hours he has averaged since the early Redis years — with heavy help from AI coding agents and GPT 5.5, while humans led the ideas, the testing, and the debugging. His more radical claim: the ds4 codebase is a working template for coding agents to adapt, for example having an agent optimize the inference kernels for your exact hardware instead of the project trying to support every configuration itself. That matches the argument we made in our 2026 opening outlook: agents are rewriting how software is developed, maintained, and distributed.
What It Means for Local AI
Three observations. First, ds4 shows that deep optimization for a short list of models can reclaim performance that generic runners leave behind; asymmetric 2/8-bit quantization brings near-frontier capability onto workstation-class Macs. Second, this is a meaningful fork in the local inference stack since llama.cpp: universal compatibility on one side, maximum performance on the other, and both will coexist for a long time. Third, the demonstration effect matters — the creator of Redis just showed that one person plus AI agents can ship an inference engine in a week, and he admits the project became popular far faster than he expected. For independent developers, that is a strong feasibility signal: the next wave of local AI may be pushed forward by small, deep, agent-rewritable projects like this one.
Sources
- antirez/ds4 (DwarfStar) — GitHub
- A few words on DS4 — antirez
- DeepSeek 4 Flash local inference engine for Metal — Hacker News
AI-assisted summary compiled from the sources above, reviewed by a human before publishing.