What Changed: From Simple Queries to Agent-Driven Complexity
A few years ago, search engine architecture was straightforward: build an inverted index, run queries, rank results, return them. But when your customers shift from human users to thousands of AI agents—each with their own language preferences, timeliness needs, and data source requirements—the picture changes dramatically.
Exa, a company building a search engine specifically for AI agents, recently shared how they tackle this complexity with a search pipeline orchestrator called Canon. The core insight: represent the search pipeline as a directed acyclic graph (DAG) so that parallelism, observability, and reliability emerge naturally.
As Exa notes in their blog post, a seemingly simple search request today can involve a graph of 20+ node types. Some users want Japanese results, others want the latest news, still others want encyclopedic references. Depending on the request, a query may hit the knowledge graph, product index, web index, or a combination of these indexes, then go through classification, localization, and re-ranking. Moreover, most of the pipeline code is now written by AI agents, which are locally correct but struggle with global constraints. At the scale of billions of requests overall, traditional imperative code with manual parallelism becomes a debugging nightmare.
Representing Search as a DAG
Exa’s solution is to define the entire search process as a DAG. Each node represents an independent unit of work (e.g., dense retrieval, sparse retrieval, fusion, content fetching), and edges represent data dependencies. The runtime automatically identifies nodes without dependencies and executes them in parallel.
According to Exa, this design provides several concrete benefits:
- Automatic Parallelism: The runtime knows which nodes can run concurrently because the graph encodes dependencies explicitly. No need to manually write
Promise.allor manage thread pools. - Durable Execution: If a node fails, the runtime can retry only that node without re-executing upstream steps.
- Introspectability: The DAG is data. It can be visualized, validated, and analyzed without reading source code.
- Decoupled Definition and Execution: The same graph definition can run synchronously in tests, distributed in production, or as a dry-run preview. Caching, retries, and tracing live in the executor, invisible to nodes.
A concrete example from Exa shows a simple pipeline with dense and sparse retrieval, fusion, and content fetching:
{
"nodes": {
"dense": { "type": "retrieve", "index": "web", "numResults": 50 },
"sparse": { "type": "retrieve", "index": "inverted", "numResults": 30 },
"fuse": { "type": "fuse", "method": "rrf", "sources": ["dense", "sparse"], "k": 60 },
"fetch": { "type": "fetch_content", "source": "dense" }
},
"root": { "node": "fetch", "output": "docs", "query": "...", "numResults": 10 }
}This graph makes dependencies explicit and allows the runtime to parallelize the two retrieval nodes automatically.
Observability Across the Search Pipeline
Before Canon, Exa’s search pipeline was a series of sequential function calls, if/else branches, and manual error handling. Swapping in a new reranker required auditing every conditional that might invoke it. Debugging why a query returned bad results meant manually tracing logs from the new reranker back, checking whether any fallbacks fired.
Exa gives a concrete example of the pain: Why did a certain query at a certain time fail to return a URL that was obviously relevant? In a traditional pipeline, the URL could have been filtered out during classification, ignored during localization, demoted during ranking, or overwritten in a concurrent branch. Finding the real cause was like finding a needle in a haystack of billions of requests.
Canon addresses this by compiling the search pipeline to a serializable graph. When debugging, you can trace the exact path and decisions made by each node: which subsystem dropped the URL? Why? Because the graph structure is explicit, you can guarantee that it matches the actual search path even when customers have specific configurations.
Figure from Exa’s blog shows many possible drop points: filtering, deduplication, freshness cutoff, domain blocking, content extraction failure, etc. With Canon, all these decisions are recorded in the graph trace.
The Runtime Design: Let Nodes Be Simple
Canon’s runtime uses a pull-based model: a node runs only when a downstream consumer asks for its value. This design yields several practical properties:
- Lazy Evaluation & Cancellation: If a client disconnects or a request times out, the runtime cancels all in-flight nodes automatically, saving compute.
- Memoization: If two downstream nodes share the same upstream node (forming a diamond dependency), that upstream node runs only once and its output is cached.
- Full Tracing: The runtime sits at every invocation boundary, automatically recording timing, inputs, outputs, and decisions. Any error thrown by a node is enriched with context about which node failed and what was happening upstream.
Crucially, nodes themselves are oblivious to these mechanisms. A retrieval node only knows how to query its index; a reranker only knows how to score documents. They don’t handle concurrency, cancellation, caching, or tracing.
Exa emphasizes this design principle: If a node makes its own network call (e.g., directly calling a ranking service), the runtime cannot cancel it, cache it, or trace it. That node becomes a black box. Therefore, all nodes must communicate through a uniform interface with the runtime.
Architecture for Agent-Written Code
Exa highlights a reality of software engineering in 2026: most code is now written by coding agents. These agents need a structure that guides them to correctness on the first attempt, not through iterative debugging.
In traditional imperative code, correctness is hidden in branch structure, execution order, and code conventions. Implicit invariants abound:
- Were the contents already fetched?
- Do we have content to rerank?
- Did we handle moderation?
Canon makes these assumptions explicit by treating the retrieval pipeline as a typed execution graph. The agent doesn’t need to guess the order of calls; it only needs to verify that the graph it produces is valid according to the schema and type system. The burden of correctness shifts from the agent’s context window to the type system, graph schema, and runtime.
Moreover, Canon enforces totality: every node must handle every possible outcome it can produce. The type checker rejects any graph with unhandled branches. This strictness eliminates a whole class of runtime surprises.
Limitations and Trade-offs
DAGs are not a universal solution. Exa explicitly notes that DAG-based orchestration fits poorly for:
- Reactive event loops: systems with no scheduled work, just state and callbacks.
- Consensus protocols: which require strict sequential ordering.
- Feedback loops: e.g., compiler passes that iterate until convergence—a DAG cannot express cycles.
If your system’s work shape doesn’t match a DAG, forcing it will add accidental complexity. Evaluate whether your pipeline has clear dependencies and partial ordering before adopting this pattern.
Concrete Takeaway
Exa’s search will only grow more complex: more search types, more customers, more agents. Canon is not a general-purpose framework but a targeted solution for systems with explicit dependencies, need for parallelism, and high observability requirements.
For product builders and AI tool learners, the key takeaway from Exa’s approach is an attitude toward complexity management: don’t try to make the system simple (it isn’t); instead, build structures that make complexity visible, traceable, and controllable. When your agents or colleagues don’t have to guess how the pipeline behaves, debugging time shrinks from hours to minutes.
If you’re designing a system that ingests multiple data sources and serves diverse customer needs, ask yourself: Is your pipeline a visualizable graph, or a chain of if/else statements that only you remember the order of?
This article is based on Exa’s public blog post “Composing a Search Engine” (April 17, 2026). Source-specific claims and examples are attributed above; practical recommendations are editorial guidance.
Sources
AI-assisted summary compiled from the sources above, reviewed by a human before publishing.