Summary: A ServiceNow research team published MosaicLeaks on the Hugging Face blog on June 18, 2026 to measure how research agents leak private facts through outbound queries. Its 1,001 multi-hop chains use the mosaic effect to reconstruct enterprise details from query strings alone. The reported results show prompting is weak, task-only RL increases leakage, and the proposed PA-DR cuts answer and full-information leakage to 9.9 percent under controlled conditions.
The MosaicLeaks benchmark landed on the Hugging Face blog on June 18, 2026, published by a ServiceNow research team alongside the identically named paper they had sent to arXiv in late May. It drills into a single question: a deep research agent that blends private local documents with web search — how much of that private material escapes through its outbound queries? The measurement is not reassuring.
The Threat Model: The Mosaic Effect
The benchmark is organized around the “mosaic effect”: taken one at a time, every web query looks innocuous, yet an adversary who follows the accumulating query log can piece private enterprise facts back together. The setup deliberately hands the attacker the weakest possible vantage point — no view of the private documents, no view of the agent’s reasoning. The sole material available is the outgoing stream of query strings.
Why insist on such a restricted vantage point? Because the query log is the artifact companies hand off with the least ceremony: it travels to search providers, observability pipelines, log aggregators, and every intermediary sitting between the agent and the web. A leakage channel needing nothing beyond read access to routine telemetry is a channel nearly every organization already exposes.
Scoring runs along three rungs of rising severity: intent leakage — working out which private questions the agent was chasing; answer leakage — recovering the answers once the log is read together with those questions; and full-information leakage — stating verifiably true private facts without being handed any questions at all.
The Dataset: 1,001 Multi-Hop Research Chains
At the core sit 1,001 multi-hop research chains: 559 for training, 98 for validation, and 344 reserved for testing, with the test chains centered on out-of-sample companies. Local and web sub-questions alternate inside each chain, and whatever one hop produces becomes the “bridge entity” without which the next query cannot be formed. DRBench-style enterprise tasks supply the local documents; BrowseComp-Plus supplies the web ones.
A worked example from the paper makes this vivid. Reading its local files, the agent learns that MediConn finished a 70% cloud migration with a January milestone; to verify the security angle, it searches the web and arrives at Microsoft’s January 2024 nation-state attack disclosure. The company name, the “70%,” and the “January” riding in that query string are exactly the fragments an eavesdropper needs to reconstruct the full picture. The agent harness is adapted from DRBench and exposes four tools — Plan, Choose, Read, Resolve — with Qwen3-4B serving as the primary test model.
Three Takeaways: Prompting Fails, Task-Only RL Hurts
Three findings carry the results. First, instructions fall short: appending a “do not leak” directive moved leakage just from 34.0% to 25.5% while success sank from 48.7% to 44.5% — the model responded by issuing fewer queries, not safer ones. Second, training on the task alone backfires: task-driven reinforcement learning lifted success to 59.3% yet pushed leakage up to 51.7%, because queries loaded with context retrieve better and spill more. Third, frequent leakage at all three levels shows up across every model family and size examined.
The second finding is worth dwelling on, since it cuts against the way “private” agents usually get shipped today. The instinct most teams follow is straightforward: make the agent better at its job, then staple a confidentiality instruction on top. MosaicLeaks puts numbers on why that instinct misfires — capability training teaches the model to pack more private context into each retrieval call, and every additional fragment carried outward is a present for whoever happens to be reading the log.
PA-DR: Baking Privacy Into the Training
The team’s answer is Privacy-Aware Deep Research (PA-DR), built from two fused signals. One is a situational task reward: each tool call is graded against its peers at the same stage and hop. The other is a learned privacy reward — a Qwen3-4B classifier rates both direct-leak and mosaic-leak risk, and whichever score is larger gets penalized.
The contrast is stark. Success comes in at 58.7%, a touch under task-only RL, but answer and full-information leakage collapse to 9.9% — lower than even the untrained baseline. Curiously, the agent issues more queries rather than fewer; what changes is their content, as revealing details such as “15%” or “2024” get stripped out. Sample efficiency stands out as well: situational rewards reach roughly 55% success on about 183k samples, versus the roughly 963k that outcome-only RL required. The team compresses its lesson into one line: “You can’t prompt privacy in. You have to train it in.”
Caveats and the Road Ahead
On scope, the authors hold nothing back: the documents are synthetic, the web corpus is fixed, three company contexts are covered, a single agent harness is used, and the task is multi-hop QA rather than open-ended research. Put plainly, this is a benchmark under controlled conditions, not a measurement of deployed systems. As of publication, the blog post also offered neither a link to a public dataset release nor stated license terms, so reproducing the work currently means going through the paper.
Even so, any agent product wiring enterprise documents into search tooling now gains the first reproducible yardstick for leakage, together with a mitigation path already demonstrated to work. And it drives home a point every agent builder in 2026 should internalize: the query log that looks harmless is an attack surface in its own right, and it only gets sharper as external retrieval turns into table stakes.