Summary: LangChain shipped AI Search in LangSmith on October 13, 2026, letting you find agent traces by describing a behavior in plain language — “did PII leak?” or “did the user get frustrated?” — instead of writing tag filters or scanning runs by hand. A fast decision model reads each normalized trace and returns matches, streaming results as they’re found. For anyone shipping agents, this changes ad-hoc debugging and product discovery from a manual slog into a query.
Why keyword search fails on traces
Agent traces hold the full story: user inputs, reasoning, tool calls, responses. The problem is that the questions you actually want to ask — did the user give up? did the agent claim success without doing the work? — don’t map to a single attribute. Tags, metadata, and full-text search all fall short because answering them requires reading the trace and reasoning about what happened.
The old workarounds were clumsy: write an evaluator ahead of time, or grind through traces manually. AI Search replaces both. You describe the behavior, structured filters narrow the candidate set, and the decision model checks each remaining trace against your description.
What you can actually look for
Per LangChain’s announcement, anything present in trace content is fair game. Their examples are worth stealing for your own backlog: an agent that said it finished but didn’t, a request that leaked PII, a user trying to cancel, a competitor mention, or users confused about onboarding. That last category is the interesting one for product people — you can hear where the product falls short in users’ own words, without building a dashboard first.
Search also composes with existing filters, so you can restrict a query to errored traces or a specific tag.
The pipeline: trajectories, budgets, and a cheap model
The engineering details matter if you’re thinking about how to build this yourself.
First, embedding-based vector search wasn’t good enough. An embedding compresses a whole trace into one vector, and that compression is lossy. LangChain points out that a trace where the agent succeeded and one where it merely claimed to succeed can land near each other in embedding space. Telling them apart takes a model that reads the words alongside your question.
Second, traces are messy and repetitive. They arrive from many SDKs — OpenAI, Anthropic, LangChain, LangGraph, the Vercel AI SDK, OpenTelemetry — each formatting messages differently, and agents typically re-send the entire conversation history on every LLM call. LangSmith’s SmithDB normalizes each session into a trajectory: every message, tool call, and result listed once. That makes one search work across heterogeneous SDKs and cuts context bloat.
Third, even deduplicated trajectories get enormous. LangChain sampled 150 conversations from Open SWE and found a couple ran past a million tokens. But the median conversation spent only about 2K of its 72K tokens on user messages and agent replies — roughly 85% was tool output like file reads and shell commands. So AI Search fills an 8K token budget in priority order: user messages and the agent’s replies first, then intermediate reasoning, tool results, and system messages. Searches stay fast because filtering, prefetching, and model checks run in parallel, and matches stream back immediately.
If you’ve been thinking about how models manage their own context budgets, this is the observability-side mirror of that problem — when the model decides to clean its own context, you get traces that are even harder to search afterward.
Known limits, stated plainly
LangChain is upfront about the gaps. The budget works when the answer lives in the user-agent exchange; an agent ignoring a tool error only shows up in the tool results, which may get trimmed. They’re exploring question-aware context selection and uncertainty signals that trigger a second pass. Subtle distinctions — “user gave up” versus “user got what they needed” — remain the hardest to classify, and they want to test a trace-tuned model on exactly those cases.
AI Search is in public beta on LangSmith Cloud (GCP US, EU, APAC) at no additional cost; an org admin enables it in settings. Practical next step: take your three most common manual “let me just eyeball the traces” moments and try them as plain-language queries. That’s the fastest way to find out whether behavior search earns a permanent slot in your debugging loop.
Sources
View the original post on X