AGENTIC COMMONSAI industry briefings

繁中EN

TopicAI Agents & WorkflowsPublished

All English articles

Grok 4.5: A Mixed Benchmark Card and a Quarter-Token Efficiency Play

On this page6 sections
  1. Positioning: an engineering workhorse trained alongside Cursor
  2. The full scorecard: wins, losses, and no sweep
  3. Training scale: hundreds of thousands of tasks, rollouts that run for hours
  4. The economics: 80 TPS, a quarter of the tokens, $2/$6
  5. Where to get it: the default model, on every plan, free for a while
  6. References

Summary: SpaceXAI released Grok 4.5 on July 16 as its strongest model to date, aimed at coding, agentic tasks, and knowledge work. The scorecard was mixed rather than a sweep, and the article highlights a nine-point DeepSWE decline after a harness change in results compiled from vendor sources. It claimed 2x token efficiency with about one-quarter output volume on SWE Bench Pro, listed pricing at $2 per million input tokens and $6 per million output tokens, put the model in Grok Build, every Cursor plan, and its billed console, and gave Grok Build and Cursor limited-time free access.

SpaceXAI released Grok 4.5 on July 16, and a single sentence covers the positioning: the strongest model the company has built to date, pointed at coding, agentic tasks, and knowledge work. The thing worth examining closely, though, is not any one score — it is the shape of the card as a whole. On several boards it takes the top spot; on others it trails by a margin nobody can miss. Layer on the claimed 2x token efficiency at one-quarter the output volume, and the question shifts: not which model is best, but which one buys the most intelligence per unit of cost.

Positioning: an engineering workhorse trained alongside Cursor

The stated capability range is concrete. At one end sits engineering work in systems languages like Rust and C/C++; at the other, producing an entire application from one prompt. This is not meant to be a generalist flagship — the design narrows deliberately onto execution. Training ran together with Cursor, and the company’s chosen phrasing, trained alongside Cursor, is more than marketing: the scoring contexts come from how a real development tool is actually used, not from how academic tests behave. The same fact explains why the model appeared in every Cursor plan on launch day rather than idling in an API console until users wired it up themselves. For teams already running coding agents hard, there is one further consequence worth flagging: the model and its harness are evaluated as a unit. Swap the model and the harness owes you a fresh round of behavior testing — confirming API compatibility proves nothing by itself.

The full scorecard: wins, losses, and no sweep

Read the official numbers as one complete set. Start with the wins: 29.0% on SWE Marathon pass@1 takes first place, ahead of Opus 4.8 at 26.0% and Fable at 24.0%. Terminal Bench 2.1 comes in at 83.3%, one point behind Fable’s 84.3% and 0.1 behind GPT 5.5’s 83.4%. DeepSWE tells a different story. Version 1.0 scores 62.0%, beaten by Fable(max) at 66.1% and GPT 5.5(xhigh) at 64.31%, yet ahead of Opus 4.8(max) at 55.75% and Opus 4.7(max) at 40.12%. Version 1.1 switches to the mini-swe-agent harness and drops to 53%, with Fable at 70%, GPT 5.5 at 67%, and Opus 4.8 at 59% all ahead of it — the only model it beats is GLM 5.2 at 44%. SWE Bench Pro lands it at 64.7%, mid-pack, behind Fable at 80.4% and Opus 4.8 at 69.2%.

Test Grok 4.5 Strongest rival on the board Where Grok lands
SWE Marathon pass@1 29.0% Opus 4.8 26.0% 1st
Terminal Bench 2.1 83.3% Fable 84.3% 3rd — 1.0 and 0.1 points behind the leaders
SWE Bench Pro 64.7% Fable 80.4% Mid-pack, upper half
DeepSWE 1.0 62.0% Fable(max) 66.1% Behind the top two
DeepSWE 1.1 53% Fable 70% Clearly behind

Read the table for the mixture, not the order. The two DeepSWE runs deserve to sit side by side: one model, 62.0% at 1.0 and 53% at 1.1, with the harness as the only variable that changed (1.1 moves to mini-swe-agent) — a nine-point swing, wider than the gap separating most of the models named above. Swap the harness, the repositories, or the scoring method, and the standings move with them; that observation covers the whole agentic benchmark ecosystem, not some flaw unique to this release. Provenance needs its own note: every figure traces back to SpaceXAI’s announcement, and the competitor numbers are drawn from their respective system cards and leaderboards — nobody reran the entire field under one roof. Sound enough for building a candidate shortlist; unsound as a purchasing ranking. The only comparison that carries real weight runs on your repository, your test suite, and your time limits.

Training scale: hundreds of thousands of tasks, rollouts that run for hours

The scale is not small: by the company’s own account, tens of thousands of NVIDIA GB300 GPUs. The RL stage spanned hundreds of thousands of tasks, concentrated on multi-step software engineering, graded through a blend of automated metrics and model-based judgment, with rollouts stretching across hours without stopping. That detail weighs more than it first appears: an agent’s unit of work is not a single generation but an entire cycle of editing, testing, and fixing. Unless the training environment admits trajectories of that length, the model never gets the chance to learn step-to-step discipline. Set against the benchmark spread, it opens a second reading: the closer a task resembles the training regime — terminal operations, marathon-length repair runs — the more reliably it wins; shift to a harness that constrains the environment and strips down the tooling, and it loses. How closely your workload matches the training shape is the surest predictor of what it will do for you.

The economics: 80 TPS, a quarter of the tokens, $2/$6

This is the part the announcement seems most invested in. On speed, the fast-model variant reaches roughly 80 TPS, which the company puts ahead of flash models. The token ledger is blunter still: across SWE Bench Pro, Grok 4.5 emitted an average of 15,954 output tokens — about one quarter (1/4.2) of the 67,020 that Opus 4.8(max) spent. The announcement additionally claims around 2x overall token efficiency and fewer than half the task steps. Both of those remain vendor assertions; no independent verification exists yet.

Put pricing on the table: $2/M input and $6/M output. If consumption runs at a quarter of a flagship competitor’s while throughput holds at fast-model level, then even without winning every board, intelligence per unit cost may be what it is actually selling. Teams running agents at volume have always treated the total cost of each finished task as closer to a purchasing decision than any leaderboard title, and Grok 4.5 packages that view as a product. Still, run the ledger both ways: if the claimed 2x efficiency and halved step count hold up, the savings arrive as tokens and round-trips. But should your tasks land in the categories it loses — long chained edits under a harness like DeepSWE 1.1’s — the added retries claw the advantage straight back. The economics hold only when your workload sits among the categories it wins.

Where to get it: the default model, on every plan, free for a while

Inside Grok Build, Grok 4.5 is already the default, with plugins for Excel, PowerPoint, Word, and Outlook — a quiet reminder that the intended scope reaches past code: knowledge work and office documents appear in the official script too, even though the evidence on offer at launch carries its weight firmly on the coding side. From day one it is available through Grok Build, every Cursor plan, and the SpaceXAI console; Grok Build and Cursor offer a limited-time free period, while the console API bills at list price. Read “limited time” literally — this is launch-window promotional pricing, not the standing rate. Build your budget on the $2/$6 list price, because when the free window closes, the invoice arrives with no cushion attached.

References

AGENTIC COMMONSOperated by PHLEGON LABS
ShareXEmail
Support us

Related reading

  1. Grok 4.6: 61 on the Intelligence Index, Frontier Again

    SpaceXAI shipped Grok 4.6 on August 12: 61 on the Artificial Analysis Intelligence Index, matching GPT-5.6 Sol, with pricing unchanged at $2 and $6 per million tokens.

    Benchmarks

  2. Epoch AI: Chinese Models Trail US Frontier by Seven Months

    Epoch AI's ECI index puts Chinese models seven months behind the US frontier on average since 2023, ranging four to fourteen months. How it is measured, and the open-weight catch.

    Benchmarks

  3. Amazon Bee Frontier and Eclipse: Chat That Turns Into Shopping

    Amazon's Bee Frontier and Eclipse listen to daily conversations and turn them into reminders, summaries, and opt-in Amazon shopping suggestions.

    AI