Summary: This comparison evaluates five academic search APIs for agent workflows, focusing on stable identity, relevant body evidence, and citation links. Its reported Firecrawl result is 53.3% recall at $0.32 per task with a 0.750 MRR in a vendor-run benchmark.
Why Academic Search Is Different for Agents
Build an AI agent that answers questions drawn from academic literature, and the retrieval layer will usually be what decides whether the answer stands up. A model can reason cleanly and still attach a citation to a paper that says nothing in support of the claim. That is what happens when the retrieval layer hands it a title and an abstract, and the model quietly supplies evidence it never actually saw.
The failure shows up in more than one shape: an attribution the source does not back, an identifier that was hallucinated, a paraphrase that has drifted away from what the paper reported. What ties these together is that the agent was asked to describe evidence it never saw.
Comparisons of tools in this space are usually written for a researcher picking something to open in a browser. The problem addressed here is a different one: which APIs can actually be wired into an agent workflow, and how each behaves once you start calling it in a loop.
For an agent, academic search is a retrieval layer that has to supply three things:
- A stable paper identity. A canonical identifier plus title, authors, and date, delivered as fields rather than text that still has to be parsed.
- Supporting content that is genuinely relevant. The body passage that speaks to the question, rather than the abstract alone.
- Links to the surrounding work. Some way of moving from one paper to what it cites, to what cites it, and to whatever sits nearby.
None of that has to arrive in a single call. What matters is that identity, evidence, and relationships are all reachable without the agent having to rebuild the academic layer itself.
General search endpoints do discovery well, yet the agent still has nothing it can quote. A general scraper returns page content and the source URL, which is enough for provenance, but the academic-specific pieces go unnormalized: DOI, arXiv ID, PMID, paper version, structured metadata, and citation relationships. Nothing stops you from rebuilding those by hand, except that inside an agent loop it turns into work repeated on every result.
The Five APIs, Side by Side
What follows groups the five APIs by the workflow each one suits, not by rank. No benchmark has been run across all five, so a single ordering would not be supportable.
Firecrawl Research Index
A retrieval index built around agent workflows in AI/ML. It launched in June 2026 and covers 3M+ arXiv papers. The capability that matters most is read mode: put a question to a paper and what comes back is scored passages from the body, not a PDF link you still have to parse. Citation-graph expansion is available as well, through the similar, citers, and references endpoints. GitHub issues, PRs, and READMEs live in a separate Developer Index.
On arXivQA, Firecrawl reports 53.3% recall at $0.32 per task together with an MRR of 0.750, which by its account places the correct paper in the top two results. The run is the vendor’s own, though the methodology has been published and can be reproduced.
Access: Keyless to start; adding a key raises the limits.
Honest take: In practice, read mode is the piece that earns its keep. The trade-off is that the AI/ML corpus covers neither economics nor journals outside arXiv, and credit-based pricing is harder to forecast than a flat per-request rate.
arXiv API
Free and stable, and a good fit for direct lookups. Where the arXiv ID is already known, this is the most direct route to clean, structured metadata. No key, no billing.
Limits: One request every three seconds, single connection at a time. No full text (abstract plus PDF link), no citation graph. Agents sensitive to freshness should note that arXiv posts scheduled announcements Sunday through Thursday, so a Friday submission may not appear before Sunday evening.
Honest take: Fine as a resolver, but restrictive inside multi-step workflows. Twenty sequential calls burn a full minute of wall-clock time before reasoning even starts.
Semantic Scholar
Coverage that runs broad across domains: 214 million papers, 2.49 billion citations, 79 million authors. Its snippet endpoint hands back query-matched text snippets drawn from open-access papers or from abstracts, tagged with snippetKind and section so you know where each snippet came from. Forward and backward citation traversal are both supported, as are batch endpoints taking up to 500 IDs per call.
Access: Free. Unauthenticated requests share 1,000 requests per second across all anonymous users; a key gives you a dedicated 1 request per second across all endpoints.
Honest take: A strong free option wherever citation traversal and cross-domain coverage matter. But snippet coverage stops at the open-access boundary, so a paywalled paper may yield nothing beyond an abstract-derived snippet. No arXivQA result has been published, so its retrieval quality relative to Firecrawl remains untested.
OpenAlex
A comprehensive metadata layer spanning domains, particularly useful for DOI resolution, structured author/institution metadata, and broad discovery. It caches full-text content (PDFs and TEI XML) for a substantial subset of the corpus, though query-ranked passages are not something it returns.
Access: Pricing denominated in dollars. Without a key: $0.10/day of usage; with a free key: $1/day. List/filter calls cost $0.10 per 1,000, keyword search $1 per 1,000, content downloads $10 per 1,000 files. Ceiling: 100 requests per second. Free public snapshot updated quarterly.
Honest take: Complements a passage-retrieval layer well, since identity and breadth are the problems it solves. Downloading a PDF, though, hands the agent the document rather than the answer-bearing evidence inside it.
Exa
A web-wide neural search API that recently added a publications index (as of July 23, 2026). Papers are covered, and so is grey literature—lab blog posts, workshop pages, release notes. A search returns full page content or targeted highlights.
Access: 10 QPS on /search; $7 per 1,000 requests for up to 10 results. The publications index is searched alongside the web index, with the two then combined and reranked.
Honest take: A strong fit when grey literature is what you need, and flat per-request pricing is easy to forecast. But there is no citation graph, and web results stay URL-oriented rather than normalized around one canonical paper identity.
Rate Limits and Freshness: the Fastest-Moving Ground in 2026
Of everything that shifted, rate limits moved the most in 2026. A two-hop citation chase is capable of issuing dozens of calls within seconds. Should a provider throttle partway through, the answer that comes back is partial while looking complete. Check the latest rate limits and billing before committing to a provider.
Freshness counts too. Where a field moves fast, preprints are the literature rather than a preview of it. How current your agent’s answers can be is shaped by arXiv’s announcement schedule, by Semantic Scholar’s unpublished live cadence, and by OpenAlex’s quarterly snapshot.
Putting It to Work: a Single-Hop Retrieval Example
The TypeScript example below uses the official firecrawl Node SDK (type-checked against firecrawl@4.32.0). It finds a paper, then pulls the passage that answers the question.
import { Firecrawl } from "firecrawl";
const firecrawl = new Firecrawl({
// Falls back to FIRECRAWL_API_KEY. Research works keyless at low volume.
apiKey: process.env.FIRECRAWL_API_KEY,
maxRetries: 3,
backoffFactor: 0.5,
});
const found = await firecrawl.research.searchPapers(
"detecting citation hallucination in retrieval augmented generation",
{ k: 5 },
);
for (const paper of found.results) {
console.log(paper.primaryId, paper.title);
}Output, the top three of five:
arxiv:2601.05866 FACTUM: Mechanistic Detection of Citation Hallucination in Long-Form RAG
arxiv:2605.27700 CiteCheck: Retrieval-Grounded Detection of LLM Citation Hallucinations in Scientific Text
arxiv:2603.27752 Retromorphic Testing with Hierarchical Verification for Hallucination Detection in RAGEvery result carries a paperId and a primaryId (the arXiv ID). Getting the supporting passage means calling read mode on the paper ID with a query:
const read = await firecrawl.research.readPaper(
"arxiv:2601.05866",
"What method does FACTUM use to detect citation hallucinations?",
);
console.log(read.passages[0].text);Where a PDF is already in hand, Firecrawl’s /parse endpoint turns it into clean Markdown in one call, which closes the last-mile problem the arXiv API leaves open.
Picking the Right Tool
No single API is best in every case. What you pick follows from your workflow:
- After quotable passages from a paper? Read mode in Firecrawl Research Index is the strongest option, but remember that its AI/ML corpus stops short of economics and of journals outside arXiv.
- Only resolving an arXiv ID you already have? The arXiv API is free and stable, but multi-step workflows are beyond it.
- Need citation traversal across domains? Semantic Scholar or OpenAlex fit better, provided you accept their rate limits or their lack of ranked passages.
- Need grey literature next to the papers? Exa is the only one of the five that covers it, and it has no citation graph.
Aemon (YC W26), which builds autonomous AI research engineers, reported from its internal benchmark that Firecrawl Research delivered the strongest recall of any provider it tested, most visibly at deeper search depths. Validation from a third party helps, but testing on your own workload is still what settles the question.
Finally, do not forget the last mile of PDF parsing. The arXiv API returns a PDF link and expects you to handle extraction. Firecrawl’s /parse endpoint converts PDFs to clean Markdown. Choose an API that covers this step and your agent’s citations become genuinely verifiable.