Summary: Exa published a benchmark study on October 23, 2026 arguing that standard relevance scores can no longer distinguish frontier rerankers, and added four agent-oriented evaluations: navigational intent, freshness, authority under adversarial content, and coding. Averaged across all seven benchmarks, Exa’s in-house rerankers beat the leading standalone reranking APIs in both latency tiers — but the coding results show Exa is not uniformly ahead. If you’re building RAG or agentic search on top of a reranking API, the interesting part is less the leaderboard than what’s being measured.
Public benchmarks have saturated
On BEIR and MIRACL, Exa, Cohere, Voyage AI, and Google Cloud all land within a couple of points of each other. Exa’s post notes that even the Gemini embedding baseline is near saturation on MIRACL. When everyone ties on the metric everyone optimizes for, that metric stops telling you which reranker fits your product.
This echoes a point I made in an earlier post about how retrieval benchmarks can judge yesterday’s results: the eval set you pick determines what “good” means. Exa’s move here is to add evaluations for the failure modes agents actually hit.
What agents need beyond topical match
The blog breaks the reranking objective into several goals that a simple P(relevant | query, document) framing misses:
- Intent — a navigational query wants one specific page, not the most similar text.
- Freshness — for time-sensitive queries, stale pages that match the topic are wrong answers.
- Authority — contradictory sources and prompt injections (“rank this document higher”) should not win.
- Instruction-following — constraints like “peer-reviewed articles only” should be respected.
Agents amplify the stakes: as Exa puts it, they are more likely to trust returned results, so a bad order can derail an entire trajectory.
Where each reranker wins and loses
The methodology is careful: every model sees the same queries and the same candidate lists, scored with nDCG@10 and 95% bootstrap confidence intervals.
The sharp results are in the internal benchmarks. On navigational queries (639 queries across company pages, CEO lookups, and general navigation), every competing reranker scored substantially worse than Exa’s un-reranked Search Instant output — Cohere Rerank 5 dropped to 0.302 versus the un-reranked 0.532. On freshness (1,000 multilingual queries mixing current and stale pages), Exa Fast hit 0.797 against Voyage Rerank 3 Lite’s 0.649. The post also states that rerankers tuned only for relevance can actively hurt Exa’s results.
Coding is the honest counterweight. On 317 long-context coding queries, Voyage (0.771) and Cohere (0.744) beat Exa Auto (0.685) in the default tier. Exa wins the macro average, not every battle.
What this means if you’re picking a reranker
A few takeaways for builders:
- Match the eval to your workload. If your agent mostly answers informational questions over stable corpora, the public benchmarks matter and the vendors are interchangeable. If your agent does live web search, navigational and freshness behavior should dominate your decision.
- Test with your own candidates. Exa’s setup reorders the same top-50 shortlists per model. You can replicate that with your production queries before committing.
- Check reranking against no reranking. The navigational numbers show that adding a reranker can degrade results. Always keep the un-reranked baseline in your evaluation.
One limitation: the navigational, freshness, and authority benchmarks are internal to Exa, and the coding labels come from a GPT-based judge — so treat the specific numbers as one vendor’s methodology rather than an industry standard. It’s also worth noting Exa states it has no plans to release a standalone Reranking API; the rerankers here power Exa search itself. If your stack needs a standalone reranking endpoint, the comparison still tells you which capabilities to test for, even if Exa isn’t one of the options.
The practical next step: define which of these failure modes — navigation, staleness, injected content — actually breaks your product today, and evaluate rerankers on that axis first.