Study guide · Safety & security · 2026

Stealing Reasoning Traces from Proprietary LLM APIs

Alexander Panfilov, David Schmotz, Ilia Shumailov, Luca Beurer-Kellner, Joachim Schaeffer, Ameya Prabhu, Jonas Geiping, Maksym Andriushchenko · arXiv preprint

Reasoning models think before they answer. Providers hide that thinking, so they send it back to the user's software as a sealed, encrypted block to pass along on the next request. This paper shows that in July 2026 those sealed blocks could be accepted by other conversations, other users and other models from the same company. A cheaper model could then be asked to read a stronger model's hidden reasoning aloud. This guide explains how the authors established that, what they measured, and what they recommend.

The paper in four sentences

When you use a reasoning model through an API, its hidden thinking comes back to you as an encrypted block. You send that block back with your next message so the model remembers what it was thinking.

The authors found that, at Anthropic, OpenAI and Google, those blocks were not tied to the conversation, user or model that made them. A block made by a strong model could be handed to a weaker model from the same company, and the weaker model could read it and write it out in plain text.

They used this to study four risks: copying a model's private reasoning, finding passwords and personal data inside reasoning people had published online, revealing harmful content that the visible answer had refused, and hiding instructions inside a block that nobody can read.

They told the companies first, the companies fixed it (the attacks stopped working by August 2026), and the paper proposes how to seal these blocks properly.

Idea · §1, §2.2

What an encrypted reasoning block is

A reasoning model writes out its working, often thousands of words, before giving a short answer. Companies keep this working hidden. They give two reasons: it is valuable (others could train their own models on it, which is called distillation), and it may contain things the final answer leaves out.

But APIs are stateless: the company's servers don't remember your conversation between requests, so your software sends the whole conversation every time. To let the model continue its earlier thinking without showing it to you, the company encrypts the reasoning and gives you a sealed block. You can't read it. You just pass it back, and the company's servers unseal it for the model.

No provider publishes exactly how these blocks are built, so the authors worked it out from how the APIs behaved (§2.2). They describe the blocks as encrypted with a tamper-proof seal (the technical term is an AEAD envelope). Their key observation was what the seal did not check: who you are, which conversation it came from, or which model made it.

Everyday pictureA cloakroom gives you a sealed envelope to hand back later. The envelope is genuinely sealed, so you can't open it. But the cloakroom will accept any genuine envelope from anyone, at any of its branches, and staff at a small branch will read the contents out to whoever handed it in.

TakeawayThe encryption itself wasn't broken. The flaw was that a sealed block could be used anywhere within the same company.

Check yourself
Why do providers send the reasoning back to the user at all, if they want it hidden?
Because the API doesn't store conversations. The client has to resend everything, so the hidden reasoning must travel with it, sealed.

Method · §2.3 · Table 1 · Fig. 2

Step one: testing where a block is accepted

The first experiment was a careful compatibility test. The authors generated blocks with their own accounts, then sent each block back in different situations and recorded whether the API accepted it. They tested three levels, each looser than the last:

  • Same or new conversation: can a block be reused in a fresh request, or moved around within one?
  • Another user: does a block made on one account work on a different account?
  • Another model: does a block made by one model work in a request to a different model from the same company?

They mapped this for Claude, GPT and Gemini models (Table 1). They also checked where in a conversation a foreign block could be placed (Fig. 2). Within each family the answer was mostly "yes, it works". Some exceptions were noted: for Claude, Fable 5's blocks were the exception, and on the GPT side only the newest series accepted all older blocks. The pattern suggested each company used a single key for every block.

A diagram of the compatibility test (§2.3), simplified. Pick where the block is sent. "Seal bound" switches on the fix the authors propose (§5.5, Appendix A), in which the seal also records who, which conversation and which model, so a mismatch is rejected. Which models accepted which blocks varied by provider and model (Table 1); this diagram shows the general pattern the paper found in July 2026.

TakeawayThe research started with a systematic map of what the APIs accepted, before any attack was attempted.

Method · §2.4–2.5 · Fig. 1

Step two: getting a cheaper model to read it out

If a block works in another model's request, the receiving model can see the reasoning inside it as if it were its own earlier thinking. The authors picked, for each company, a decoder: a cheaper model that accepted the stronger models' blocks and could be persuaded to copy that reasoning into its visible answer. For Claude this was Haiku 4.5, for GPT it was GPT-5.6 Luna, and for Gemini it was Gemini Robotics 1.6.

Why a weaker model? Attacking the strong model directly means beating its refusals plus the company's input and output filters. The weaker model often has fewer protections, and it is cheaper to run at scale. Even so, for GPT the authors needed several attempts and workarounds for anti-distillation safeguards.

Test material (§2.5): to get blocks they could study, they asked the strong models hard problems from three benchmarks: AIME 2025 (maths competition), Codeforces (programming contests) and Humanity's Last Exam (expert questions).

How they judged success: the hidden reasoning can't be checked word for word, because nobody outside the company has the original. But the API reports how many "thinking tokens" the model used. So the authors compared that reported count with the length of the reasoning the decoder wrote out. If the two track each other, the decoder is probably copying the whole thing, not summarising or inventing. Figure 1 plots this for 120 Codeforces problems, and the authors say the counts "track each other closely across all tested models".

How the paper's main quality check works (Fig. 1, §2.4). Each dot would be one problem. Dots on the diagonal mean the decoded text is as long as the reasoning the API says happened. The grey dots are made up to show the idea, not the paper's data; move the sliders to place your own dot. The authors stress that this is a proxy: matching length makes a full copy likely but doesn't prove it.

TakeawayWith no original text to compare against, the authors measured success indirectly: does the decoded reasoning have the length the provider itself reported?

Check yourself
Why can't the authors simply compare the decoded text with the real hidden reasoning?
Only the provider can see the real reasoning. The reported token count is the one piece of ground truth an outsider gets, so it serves as the yardstick.
What would a decoded text much shorter than the reported count suggest?
That the decoder summarised or skipped parts rather than copying the whole trace.

Experiment · §3 · Figs. 3–4

Risks from your own blocks: copying and hidden content

The paper sorts its experiments by who holds the block. A first-party attacker makes blocks with their own account, then reads them out with the cheaper model.

Copying reasoning (§3.1). Full reasoning is a much richer training signal than final answers, so readable traces would make distillation easier. The authors estimate that decoding 10,000 long traces would cost about $720 at Haiku 4.5 prices. They also ran a probe on open models: they gave each model a short piece of decoded Claude reasoning to continue (Fig. 3, Appendix B). Some, such as Kimi-K3, then reasoned and wrote noticeably more like Claude. The authors say plainly that this "cannot establish a causal claim" that those models were trained on Claude's reasoning.

Content hidden behind a refusal (§3.2). Models are trained to keep harmful details out of their answers, but not necessarily out of their thinking. In one example (Fig. 4), a harmful-style test question got a harmless visible answer, while the decoded reasoning contained information the answer had left out. This is a single illustration, not a measured rate.

TakeawayThe first-party experiments mostly show what becomes possible. Apart from the cost estimate, they are demonstrations, not large measured studies.

Experiment · §4.1 · Figs. 5–6 · Table 4

Risks from other people's blocks: secrets in public logs

A third-party attacker uses blocks that other people made. Where would they find them? Developers often publish logs of AI agents at work, for example on GitHub or Hugging Face. Those logs still contain the sealed reasoning blocks.

The authors collected 6,708 such public agent logs from Claude, GPT and Gemini models. They decoded every block they found, 315,320 in total. An AI model then labelled each decoded trace for private information such as passwords, API keys or addresses, in two stages (Appendix D).

About 0.3% of blocks (1,028) contained at least one leak, and 4.9% of logs (328) leaked at least one real sensitive item. In logs from real users they counted, among other things, 62 API keys, 33 passwords and 24 access tokens (§4.1, Fig. 6; the figure's caption says it covers all sources, so the paper is not fully consistent about scope). The paper's abstract summarises the findings as 367 pieces of personal information and 182 credentials.

A common cause was ironic: asking an agent to clean up or anonymise a log before publishing it. To decide what to remove, the model re-read the history and wrote the secrets out in its hidden reasoning. In 64 cases the secret appeared only in the reasoning, not in the visible log (Table 4).

Ethics (§5.3): all decoding and labelling ran in an isolated environment, the recovered secrets were deleted after counting, and the platforms and providers were told before publication.

Numbers as reported in §4.1 and Fig. 6 of arXiv v1. In the first view the bars are on a log scale so that 6,708 logs and 315,320 blocks fit on one chart; the second view is to scale. The labels come from an automated AI judge, so counts may include some errors.

TakeawayThis was the paper's largest measured experiment. It showed that reasoning published "safely" sealed was readable and sometimes held secrets the visible log didn't show.

Check yourself
Why could asking an agent to remove secrets make things worse?
To find the secrets, it restates them in its hidden reasoning. That reasoning stays in the published log as a sealed block, which this flaw made readable.

Experiment · §4.2

Hidden instructions inside a block

The flaw also works in the other direction. If a model accepts a foreign block as its own earlier thinking, someone could plant a block carrying an instruction. Anyone who later continues that conversation, for example by reusing a shared agent log, gives their model a "memory" it never had. Neither the person nor a safety monitor reading the visible text can see the instruction, because it is sealed.

The authors showed this in two demonstrations. In both, a block made by a smaller model was carried into a larger one, and the larger model followed the planted behaviour on an unrelated task. One ran inside a long, realistic coding-agent session (PostTrainBench in the Claude Code tool). The paper reports these as proofs of concept, with no success rate.

Everyday pictureImagine someone could slip a note into your diary in your own handwriting. You would read it later and trust it, because it seems to be your own thought.

TakeawayUnreadable reasoning that is also replayable is a hiding place for instructions that monitoring of visible text can't catch.

Outcome · §5.2, §5.5 · Appendix A

Disclosure, fixes and limits

Disclosure. Before publishing, the authors reported to the major model providers, Microsoft and Hugging Face. They note that an independent researcher, Green, had reported that blocks were interchangeable in May 2026. According to the paper, providers did not acknowledge security implications at that time. After this paper's report, all providers acknowledged it, and the authors "were unable to launch the same attacks". By August 2026 the Figure 1 results were no longer reproducible. The project cost about $30,000 in API fees.

Proposed fixes.

  • Keep the reasoning on the company's servers and give users only a reference number.
  • Bind each seal to the user, the conversation and the model, and chain each block to the one before it, so blocks can't be moved.
  • Block cross-model use at the gateway.
  • Watch for the same block appearing in many conversations.
  • Revoke leaked blocks.
  • Train models to refuse requests to transcribe their reasoning.

The authors add a deeper point: any model that can read its own sealed reasoning can, in principle, be talked into revealing it, so sealed reasoning is only "semi-hidden" (§5.5).

Limits (§5.1): the tests reflect the APIs of early July 2026, which have since changed. The decoders are random, and without the original text, completeness can only be estimated. The public-log scan was a targeted demonstration, not a full audit.

TakeawayThe study followed the usual responsible-disclosure path: find, measure, report privately, wait for fixes, then publish with recommendations.

Check yourself
If you publish AI agent logs, what does the paper recommend?
Strip the sealed reasoning blocks out before sharing, especially if any secrets or private data might have come up in the session (§5.4).

How this connects to the rest of the archive

Hidden reasoning exists because of chain-of-thought prompting and reasoning models trained with reinforcement learning, such as DeepSeek-R1. Copying a model by training on its outputs is knowledge distillation. The hidden-instruction risk is a new kind of prompt injection.

Follow the idea

This is the newest step on its timeline so far.