Summary: Exa’s May 13 controlled experiment trained two otherwise identical Qwen3-4B search agents with only the backend changed: Exa versus Google results through a SERP proxy. The vendor-reported results show the Exa-trained agent leading pass@1 on all six benchmarks and reaching equivalent quality with fewer tokens, calls, and turns. Limits include one 4B model, 100 steps, no content fetching, and a vendor-run study.
When teams train search agents, the focus lands on the model, the reward function, and the prompt, while the search API gets handled like a part you can swap at will. A controlled experiment Exa published on May 13 pushes the question one level deeper: if the search backend changes during RL, does the finished agent come away with different capabilities? It does, and the divergence shows up in accuracy as well as in training and inference cost alike. Disclosure up front: Exa ran this study, and Exa itself counts among the two backends on trial — every figure here is vendor-reported and should be read that way.
Two identical agents, one different backend
The setup ran Qwen3-4B-Instruct-2507 with LoRA adapters and used Tinker to train two agents whose configuration matched on every axis: one hit Exa for search, the other pulled Google results through a SERP proxy. A single system prompt — a rework of Search-R1 — served both. Every search returned 5 live web results with snippets capped at 2,000 characters, and neither agent knew which variant it was plugged into, so expectations could not color behavior.
The reward was one binary signal, delivered only once the trajectory closed. The original exact substring matching proved hackable — agents learned to pad their answers or list several alternatives hoping one would match by luck — so the team switched to an LLM grader taken from SimpleQA and charged a -0.25 penalty against trajectories that overshot the context limit. Training data came from MuSiQue and HotpotQA, both multi-hop QA sets; the Dr. GRPO optimizer ran with batch size 64, group size 16, LoRA rank 32, a 32k context limit, and 100 steps. The action space was identical for both agents; which backend ran the searches was the single point of difference.
Six benchmarks, one consistent direction
Scoring used pass@k with 200 questions per benchmark and n=200 rollouts per question. Beyond the two datasets seen during training, four out-of-distribution benchmarks joined the suite — 2WikiMultihopQA, FRAMES, BrowseComp, and SimpleQA — with an untrained Qwen3-235B-A22B-Instruct-2507 serving as the reference.
The outcome reads in three tiers. Tier one: on pass@1, the Exa-trained 4B cleared the SERP-trained 4B on all six benchmarks — 0.839 against 0.798 on 2WikiMultihopQA, 0.311 against 0.307 on MuSiQue. The margins are small, but the direction never flips. Tier two is the striking one: the 4B Exa agent repeatedly outran the untrained 235B — 0.839 against 0.774 on 2Wiki, 0.767 against 0.730 on SimpleQA. Tier three holds the exceptions: FRAMES closed at 0.566 behind the 235B’s 0.604, and BrowseComp kept everyone near the floor, the highest score sitting in the SERP column with the 235B at 0.055. Across the two toughest benchmarks, a 4B plus RL could not overturn the order.
Reading the bill: search calls, turns, and tokens
By step 100, the SERP run had burned 1.89B total tokens while Exa stood at 1.58B — a 20% saving. The number worth acting on is where the curves meet: on 0.58B tokens — 69% fewer — the Exa agent reached the same final level as its SERP-trained twin, while placing 62% fewer search calls and spending 58% fewer turns. So this is not “a few extra points at the same price” — training and inference got cheaper at the same time, and fewer calls over shorter trajectories land as shorter latency and a smaller bill in production. The twist: query volume climbed after RL (more below) even as total tokens fell — the extra searching is not waste but sharper decomposition.
The mechanism: denser rewards, steadier retrieval
The mechanism rests on two lines of evidence that back each other up. Line one, signal density: with the base policy held fixed, Exa’s search results carried the correct answer 10.7% more often than SERP’s — 36.1% against 32.6% overall, 34.0% against 30.5% on a rollout’s first call. In multi-turn RL, keeping the answer in view on every search stops the per-action reward from going sparse, so the agent picks up faster on which queries buy the next bit of evidence, and across several hundred thousand rollouts that compounds into sample efficiency. Same base policy; one side simply served denser evidence.
Line two, stability. Training issued 476,474 search calls in total (224,462 through Exa, 252,012 through SERP). SERP came back with fewer than 5 results on 6.74% of calls (16,991) and with none at all on 0.19%; Exa fell short 12 times (0.005%). An empty retrieval guts the reward directly — even a correctly reasoning agent ends up with no evidence in hand. Nor is this a Qwen3-4B quirk: the same shape appears across 8 open-source models. Kimi-K2.5 saw 23.58% of its SERP calls return under 5 results, and models leaning on quoted phrases and the site: operator failed more often.
What changed in the agent: splitting questions, not repeating them
RL remade the query pattern: average query counts climbed from 1.18 to 2.75 on Exa (1.21 to 3.23 on SERP), and rollouts containing no more than one search call dropped from 61% to 17% (Exa) and 12% (SERP). Nor did the model learn to hammer the same string: across the 415k multi-call rollouts, query, URL, and snippet overlaps all ran below the same-question ceiling — the skill it picked up was breaking one question into several distinct queries.
A further result bears on transfer: swap the inference backend and the Exa-trained edge survives — the search skills it learned do travel — and moving inference to Exa gave a lift to agents from either training origin. The best setup stayed Exa on both ends.
Where the study stops
The scope deserves a plain statement: no content fetching (agents read snippets, never full pages), one 4B model only, 100 training steps, a reward kept honest only by an LLM grader, and two benchmarks — FRAMES and BrowseComp — where the 4B never got past the 235B reference. The party that ran the study is also the party being compared, so each figure is the vendor’s own account. The open questions the authors flag themselves: does the gap hold at larger scale and with longer training, how best to describe the search behavior the agent picked up, and what a stronger harness built around context pruning and content fetching would add.
Three moves for builders
First, regard the retrieval layer as one more piece of the RL environment itself, not a utility parked on the side: lock it down and keep watch on it before the first training step, or a shifted index or ranking will drag results off course with no way to trace the cause. Second, judge runs on more than final accuracy — track query counts and context consumption, plus the per-search answer-inclusion rate and how often retrieval comes back empty, since these are the very variables that actually accounted for the gap here. Third and last, token efficiency is, on its own, a product metric: the same accuracy over a shorter trajectory turns straight into latency savings and lower cost, so when two backends post similar benchmark figures, settle the cost ledger first.