In the reasoning-model era, every major API returns the model’s thinking process to developers. Anthropic, OpenAI, and Google package it as a signed or encrypted block that cannot be read directly. A paper posted to arXiv on August 10 and discussed widely on Hacker News on August 11 — “Stealing Reasoning Traces from Proprietary LLM APIs” — punches an unexpected hole in that design: the thinking blocks are encrypted, but they are portable.
The paper comes from a team spanning MATS Research, the ELLIS Institute Tübingen, the Max Planck Institute for Intelligent Systems, Snyk, and other institutions. The most ironic part of their attack: nobody ever attacks the strong model itself. Everything happens on the vendor’s weaker sibling model.
How It Works: Two API Calls
The recipe is simple. Take the encrypted thinking block produced by a strong model — Anthropic’s claude-opus-4-8, for example — and replay it, unchanged, into a conversation with the same vendor’s weaker, less-guarded sibling model (claude-haiku-4-5). Then jailbreak the weak model and ask it to transcribe that “unreadable input” verbatim. Two calls later, the strong model’s reasoning is in plaintext.
The OpenAI side works the same way: GPT-5.2 Codex thinking traces were decoded with GPT-5.6 Luna, and Google’s API showed the same problem. To confirm the transcripts are real rather than hallucinated, the team tested 120 Codeforces problems: the length of the recovered reasoning closely matches the hidden thinking-token counts the API itself reports.
Scanning Public Trajectories: Secrets Inside Thoughts
Beyond the attack, the paper includes a measurement closer to everyday reality. The team scanned 6,708 public agent trajectories and reconstructed 315,320 thinking blocks, finding 704 distinct privacy artifacts inside them: 62 API keys, 33 passwords, 24 access tokens, and 30 personal email addresses. Sixty-four of those artifacts appeared only inside the reasoning, never in the visible output. (The arXiv abstract reports a narrower subset — 367 PII artifacts and 182 credentials — from the same scraped blocks.)
In other words, chains of thought are becoming a new boundary for data leakage: users and agents write secrets into a place “only the model should see,” and that place has now been proven readable.
Four Attacks from One Design Flaw
The paper maps out four attack lines. First, bypassing anti-distillation protections: reasoning traces were demonstrated to be extractable from Anthropic, OpenAI, and Google, rendering protections meant to keep traces out of rival training pipelines largely symbolic. Second, mass private-data extraction — the scanning results above. Third, hazardous content can hide inside a thinking block while the visible answer stays perfectly benign, invisible to anyone auditing outputs. Fourth, invisible prompt injection: instructions can ride into a conversation inside the thinking block itself.
A supplementary experiment is worth remembering: prefilling reasoning content into Moonshot’s Kimi-K3 shifts its answers. These blocks are not just text — they have real control over model behavior.
Root Cause and Fixes: It Is Not About Crypto Strength
The core of the vulnerability is not broken cryptography but a design decision: thinking blocks are interchangeable across conversations, users, and models — effectively bearer tokens. After responsible disclosure to the affected providers, the authors propose cryptographic and system-level mitigations, with the direction of binding each thinking block to a single conversation, user, and model. The logic is the same as a session token: a block minted for one conversation should fail validation anywhere else. The paper also flags a detail that is easy to miss: the reasoning summaries APIs return can misrepresent the actual thinking — and any oversight mechanism built on those summaries can be manipulated.
What It Means for Developers
Three practical takeaways. First, do not put secrets into context you send to a model: they can end up written into reasoning and later reconstructed. Second, when you publish agent trajectories — logs, traces, demo data — scrub the thinking blocks along with everything else. Third, when evaluating vendors, ask one more question: how are thinking blocks signed and bound to a conversation? In the reasoning-model era, the thought process itself is an asset that needs access control.
Sources
- Stealing Reasoning Traces from Proprietary LLM APIs — stolen-thoughts.com
- Stealing Reasoning Traces from Proprietary LLM APIs — arXiv
AI-assisted summary compiled from the sources above, reviewed by a human before publishing.