On March 11, 2026, NVIDIA released Nemotron-3-Super-120B-A12B under its open model license: 120 billion total parameters, 12 billion active, and a one-million-token context window. It is not chasing the “smartest model” title. Third-party benchmarks put its overall intelligence score well behind Gemini 3.1 Pro and GPT-5.4. But at 478 output tokens per second, it is the fastest model Artificial Analysis has ever measured. NVIDIA’s bet is explicit: in the agent era, the bottleneck is not single-answer intelligence, it is the inference cost that accumulates across long, multi-step, tool-calling workloads.
The timing was deliberate. This is the second model in the Nemotron 3 family, arriving 86 days after December’s Nemotron 3 Nano (30B), and it landed in the week before GTC.
An Open-Source Move the Week Before GTC
The New Stack spotted a telling detail: the original roadmap called for a 100B model with 10B active parameters, and the final version grew to 120B/12B — which lines up neatly against OpenAI’s gpt-oss-120B. The open-model market in 2026 is a head-on fight, one we flagged in our opening-of-year outlook when the open-source routes diverged. NVIDIA’s answer is to out-open the competition: not just weights, but datasets and training environments too.
Hybrid Architecture: Mamba Plus Latent MoE
The differentiation is architectural:
- Latent MoE engages four experts for the cost of one
- Hybrid Mamba-Transformer layers deliver 4x better memory and compute efficiency than a pure transformer stack, suited to tracking context over long tasks
- Multi-Token Prediction roughly triples inference speed
- A 1M-token window that can swallow a whole codebase in one pass
Every one of these choices points at the same problem: context explosion in agent workloads. NVIDIA estimates that agents can generate up to 15x more tokens per step than plain chat, and reasoning models add a “thinking tax” that compounds across multi-turn tool calls.
Efficiency Numbers and Third-Party Benchmarks
The official numbers are loud. Compared with the previous-generation Nemotron Super, throughput is up to 5x higher and accuracy 2x better. On Blackwell hardware with NVFP4 quantization, it runs up to 4x faster than FP8 on Hopper with no accuracy loss. NVIDIA also claims 7.5x inference throughput over Qwen3.5-122B, and says the model powers NVIDIA AI-Q to No. 1 on both DeepResearch Bench and DeepResearch Bench II.
Third-party numbers are cooler. Artificial Analysis scores it 36 overall — slightly above gpt-oss-120B at 33, but behind Gemini 3.1 Pro and GPT-5.4 (both 57), and trailing Qwen3.5-122B, DeepSeek V3.2, and GLM-5. The speed is real, though: 478 tokens per second of output is the fastest the organization has ever measured, with gpt-oss-120B second at 264. This is a model that deliberately trades intelligence for throughput — and when a task runs hundreds of tool calls, that trade can be exactly the right one.
What Is Actually in the Open Release
NVIDIA is opening more than most rivals do:
- Model weights under the NVIDIA Nemotron Open Model License
- More than 10 trillion tokens of pre-training and post-training datasets, including synthetic data generated from frontier reasoning models
- 15 RL training environments, evaluation recipes, and a full technical report
- A complete fine-tuning path through NVIDIA NeMo
Getting it is straightforward. Direct channels include build.nvidia.com, Hugging Face, OpenRouter (currently free), and Perplexity. Enterprise access runs through Google Vertex AI and Oracle Cloud, with AWS Bedrock and Azure listed as coming soon. It also ships as an NIM microservice that can be self-hosted on CoreWeave, Crusoe, Nebius, Together AI, and other platforms.
What It Means for Developers and Platform Teams
First, “smart enough, extremely fast, and self-hostable” is a combination with real use cases. The adopter list already includes Perplexity, CodeRabbit, Factory, Greptile, Palantir, Cadence, Dassault Systèmes, and Siemens, with workloads concentrated in whole-codebase loading, long-document financial analysis, and reliable tool calling. Second, the release includes datasets and RL environments, so fine-tuners can reproduce the training path instead of receiving only final weights — a substantive difference for teams doing domain specialization. Third, a larger Nemotron 3 Ultra (550B total parameters) is still waiting in the wings. If it inherits the same philosophy, the open camp will keep measuring itself against closed frontier models in efficiency, not in leaderboard scores.
Sources
- Nemotron 3 Super for Agentic AI — NVIDIA Blog
- NVIDIA Launches Nemotron 3 Super, a 120B Open Model — The New Stack
- Nemotron-3-Super-120B-A12B Model Card — NVIDIA NIM
AI-assisted summary compiled from the sources above, reviewed by a human before publishing.