AI news and explainers at 7 AM weekdays, plus a Sunday weekly at 8Get it in your inbox

METAL LAB

Stealing Reasoning Traces from Proprietary LLM APIs

arXiv:2608.098672026-08-09

Even when AI companies hide a model's internal reasoning behind encryption, a weaker sibling model can be tricked into reading it out loud

Providers like Anthropic, OpenAI, and Google now hide a reasoning model's step-by-step internal thoughts by returning them to users as encrypted blocks instead of plaintext. This paper found that these encrypted blocks can be replayed across sessions, users, and even different models within the same provider, and used this to feed a strong model's encrypted reasoning into a weaker, less-guarded sibling model that then transcribes it verbatim in plaintext. Applying this to 315,320 reasoning blocks scraped from public repositories recovered 367 personal-data artifacts and 182 credentials.

METAL LAB explanatory visual

How the weak-model decoding attack works

Evidence statusMeasured results reported

  1. Query the strong modelAsk a heavily-guarded model like Opus 4.8 a question and capture the encrypted reasoning block (signature) it returns.
  2. Move the encrypted blockTake that encrypted block, originally tied to a different session, user, or model, and insert it into a request to another model in the same provider's family.
  3. Hand it to a weaker modelFeed the block to a less-guarded sibling model like Haiku 4.5 and ask it to transcribe its own thinking.
  4. Plaintext comes outThe weaker model decrypts and outputs the strong model's hidden reasoning verbatim in plaintext.
  5. Four abuse pathsThis enables distillation of proprietary reasoning, extraction of leaked PII/credentials, exposure of hidden harmful content, and invisible prompt injection.
An explanatory diagram made by METAL LAB, not a figure supplied by the paper's authors.

What they did

  1. Modern reasoning models think through problems step by step before answering, and providers encrypt this internal chain-of-thought before sending it back to the client to protect IP and limit information leakage.
  2. The researchers discovered that these encrypted reasoning blocks are portable: they can be replayed across different sessions, different users, and even different models within the same provider's ecosystem.
  3. By capturing an encrypted reasoning trace from a strong, well-guarded model (e.g., Claude Opus) and feeding it into a weaker, less-guarded sibling model (e.g., Claude Haiku) with a request to transcribe its own thinking, they got the weaker model to output the strong model's hidden reasoning verbatim, without ever jailbreaking the strong model directly.
  4. Applying this technique to 6,708 real agent session logs scraped from GitHub and Hugging Face, they decoded 315,320 reasoning blocks and recovered 367 PII artifacts and 182 credentials, including 62 API keys, 33 passwords, and 30 personal emails from genuine user sessions.
  5. They also showed that harmful information can remain hidden inside a model's reasoning even when its visible, final answer safely refuses the request, and this hidden content can be extracted via the same technique.
Figure 1: Decoding reasoning traces in Anthropic, OpenAI and Google APIs. Top: Reasoning-trace extraction in two API calls. An Opus 4.8 request (top left) returns a signed thinking block along with a thinking summary. Sending just the thinking signature from Opus 4.8 to a Haiku model and requesting it to output its own reasoning in <thinking-copy> tokens makes Haiku transcribe the Opus 4.8 hidden reasoning (top right). Bottom: Extracted traces closely track the number of generated thinking tokens. We evaluate each model on 120 Codeforces programming problems and record the number of thinking tokens generated by the source model, as reported by the API (x-axis). We then reconstruct the reasoning trace from its signature, pass it as an input message to the same model that generated encrypted reasoning, and measure its API-reported token count (y-axis).
Figure 1: Decoding reasoning traces in Anthropic, OpenAI and Google APIs. Top: Reasoning-trace extraction in two API calls. An Opus 4.8 request (top left) returns a signed thinking block along with a thinking summary. Sending just the thinking signature from Opus 4.8 to a Haiku model and requesting it to output its own reasoning in <thinking-copy> tokens makes Haiku transcribe the Opus 4.8 hidden reasoning (top right). Bottom: Extracted traces closely track the number of generated thinking tokens. We evaluate each model on 120 Codeforces programming problems and record the number of thinking tokens generated by the source model, as reported by the API (x-axis). We then reconstruct the reasoning trace from its signature, pass it as an input message to the same model that generated encrypted reasoning, and measure its API-reported token count (y-axis).
Figure 3: Prefilling Kimi K3’s reasoning changes the style of its visible responses. In this example, we observe that prefilling a small number of Claude-generated reasoning tokens into Kimi K3’s reasoning trace shifts its final output to closely match Claude’s. In all cases, the visible response is free-form generation and is not itself prefilled. We quantify this phenomenon in Appendix B.
Figure 3: Prefilling Kimi K3’s reasoning changes the style of its visible responses. In this example, we observe that prefilling a small number of Claude-generated reasoning tokens into Kimi K3’s reasoning trace shifts its final output to closely match Claude’s. In all cases, the visible response is free-form generation and is not itself prefilled. We quantify this phenomenon in Appendix B.
Table 1: Cross-model compatibility of encrypted reasoning. As per July 2026. Row: the source model that produced the encrypted reasoning block; column: the target model receiving the injected reasoning. A ✓ indicates that, for this combination, the target model interacts with the injected thought. Claude: the thinking traces of any model can be replayed by any other, except Fable 5’s thoughts. GPT: the GPT-5.6 series can replay the traces of all earlier model generations. Gemini: the thinking traces of any model can be replayed into any other.
ClaudeGPTGemini
Source / TargetF5O4.8S5S4.6S4.5H4.5Source / Target5.6s5.6t5.6l55-m5-nSource / Target3.1P3PRob3.5F3F
Fable 5✓✗✗✗✗✗GPT-5.6-sol✓✓✓✗✗✗Gemini 3.1 Pro✓✓✓✓✓
Opus 4.8✓✓✓✓✓✓GPT-5.6-terra✓✓✓✗✗✗Gemini 3 Pro✓✓✓✓✓
Sonnet 5✓✓✓✓✓✓GPT-5.6-luna✓✓✓✗✗✗Gemini Robotics 1.6✓✓✓✓✓
Sonnet 4.6✓✓✓✓✓✓GPT-5✓✓✓✓✗✗Gemini 3.5 Flash✓✓✓✓✓
Sonnet 4.5✓✓✓✓✓✓GPT-5-mini✓✓✓✗✓✗Gemini 3 Flash✓✓✓✓✓
Haiku 4.5✓✓✓✓✓✓o4-mini✓✓✓✗✗✓Gemini 3.1 Flash Lite✓✓✓✓✓
Figure 4: Reasoning exposes harmful information that is absent from the final output. We paraphrase a HarmBench query (26) to elicit longer reasoning from Opus 4.8. Consistent with prior findings on the chain-of-thought of open-weight reasoning models (45), Opus 4.8’s decoded reasoning reveals harmful information that could enable misuse uplift, even though its final answer remains benign.
Figure 4: Reasoning exposes harmful information that is absent from the final output. We paraphrase a HarmBench query (26) to elicit longer reasoning from Opus 4.8. Consistent with prior findings on the chain-of-thought of open-weight reasoning models (45), Opus 4.8’s decoded reasoning reveals harmful information that could enable misuse uplift, even though its final answer remains benign.
Figure 5: Decoded reasoning contains privacy artifacts. We present two qualitative examples of decoded opaque reasoning blocks published online that contain privacy-sensitive information. Left: GPT-5.2 Codex recalls the API keys that must be removed before publishing a repository on GitHub. We mask the final five characters of each key as XXXXX. Right: Claude Sonnet 4.6 reasons over the private data of a synthetic persona, Alex Green, while handling a flight-booking task in a ClawBench (44) rollout. The Alex Green persona is a synthetic benchmark identity, not a real person: https://huggingface.co/datasets/TIGER-Lab/ClawBench/blob/main/shared/alex_green_personal_info.json. We provide more examples in Section D.3.
Figure 5: Decoded reasoning contains privacy artifacts. We present two qualitative examples of decoded opaque reasoning blocks published online that contain privacy-sensitive information. Left: GPT-5.2 Codex recalls the API keys that must be removed before publishing a repository on GitHub. We mask the final five characters of each key as XXXXX. Right: Claude Sonnet 4.6 reasons over the private data of a synthetic persona, Alex Green, while handling a flight-booking task in a ClawBench (44) rollout. The Alex Green persona is a synthetic benchmark identity, not a real person: https://huggingface.co/datasets/TIGER-Lab/ClawBench/blob/main/shared/alex_green_personal_info.json. We provide more examples in Section D.3.
Table 2: Leakage vectors mapped to protections and processes. Each tick states what the mitigation supplies; the final column records the operational steps that deliver it.
#IssueWhat the mitigation providesMitigation
1Cross-user leakage✓ User-identity binding ✓ Stateless verification ✓ Immediate mismatch rejection1. Embed user_id in AEAD associated data at issuance. 2. On replay, compare bound identity to authenticated caller. 3. Reject the envelope on any mismatch.
2Cross-session leakage✓ Session + predecessor binding ✓ Ordinality (P1) under compaction ✓ Native fork / compact / downgrade support ✓ Dramatically reduced blast radius1. Hash-chain each envelope to session_id and its predecessor (Eq. 1). 2. Enforce ordinality server-side. 3. Retain only Merkle roots after compaction so surviving spans stay verifiable.
3Legacy public/enterprise datasets✓ Permanent undecodability of pre-fix signatures ✓ Clean cryptographic separation from new material1. Rotate every pre-fix signing key. 2. Refuse to decode any envelope under a retired key ID. 3. (Optional) Offer identity-verified re-signing for enterprise archives.
4Backwards compatibility✓ Zero-break migration path ✓ Bounded dual-format window ✓ Identity-verified re-issuance1. Accept both legacy and context-bound envelopes during a fixed deprecation window. 2. Expose an opt-in batch re-signature endpoint. 3. Re-issue only after confirming the requester owns the original session.
5Model-level compliance✓ Closure of residual gaps beyond cryptography ✓ Resistance to transcription / replay jailbreaks1. Post-train models to recognise transcription-style prompts (e.g., <thinking-copy>). 2. Refuse the request irrespective of envelope validity.
6Nonce predictability✓ Critical since key is shared between users ✓ Cryptographic foundation for every binding above ✓ Collision- and forgery-resistance at provider scale1. Draw a high-entropy nonce from a CSPRNG for every block. 2. Enforce server-side uniqueness before the envelope is issued.
Figure 6: Distinct artifacts recovered from reasoning blocks scraped from publicly available user-posted traces, grouped into three headline categories (all sources; see Appendix D).
Figure 6: Distinct artifacts recovered from reasoning blocks scraped from publicly available user-posted traces, grouped into three headline categories (all sources; see Appendix D).
Figure 7: Illegible GPT-5 reasoning. GPT-5 reasoning decoded with GPT-5.6 Luna; the ratio of decoded to API-reported thinking tokens is 1:1. Compared to Gemini and Claude, obfuscated reasoning appears more common in GPT models, including GPT-5.6 Sol, with artifacts similar to those previously reported by 36.
Figure 7: Illegible GPT-5 reasoning. GPT-5 reasoning decoded with GPT-5.6 Luna; the ratio of decoded to API-reported thinking tokens is 1:1. Compared to Gemini and Claude, obfuscated reasoning appears more common in GPT models, including GPT-5.6 Sol, with artifacts similar to those previously reported by 36.
Table 3: Best-of-k n-gram overlap with the source model’s visible answer. We measure overlap with the first 100 tokens of the visible answer produced by the model supplying the reasoning prefill, using the commong n-gram intersection. For each problem, we compute best-of-k overlap for k∈{1,10,50,100}, average across these four values, and then average across problems. Δ denotes the difference between the prefilled and unprefilled conditions. We report two-sided paired t-tests on the per-problem differences. In the lower block, Kimi-K3 and Inkling supply prefills to each other, so neither model receives proprietary reasoning. After Bonferroni correction, only Kimi-K3 mean differences remain significant; none of the control comparisons do.
ModelPrefillCategoryPrefilledControlΔp
Kimi-K3Opus 4.8STEM0.3050.160+1.5×10−11.7×10−5
Kimi-K3Opus 4.8non-STEM0.2890.203+8.6×10−26.3×10−6
InklingOpus 4.8STEM0.2170.205+1.2×10−27.4×10−2
InklingOpus 4.8non-STEM0.2410.239+2.1×10−35.5×10−1
Kimi-K3InklingSTEM0.3590.337+2.2×10−21.2×10−1
Kimi-K3Inklingnon-STEM0.2720.263+9.4×10−35.5×10−1
InklingKimi-K3STEM0.4140.411+2.2×10−37.2×10−1
InklingKimi-K3non-STEM0.3200.306+1.4×10−23.3×10−2
Figure 8: An example of summary unfaithfulness. For the AIME 2025 Problem 14, we compare the summary of Claude Opus 4.8’s thinking returned by the API (left) with our decoding of the thinking block’s signature (Section 2.4). Decoding reveals that the model states the correct answer before attempting to solve the problem.
Figure 8: An example of summary unfaithfulness. For the AIME 2025 Problem 14, we compare the summary of Claude Opus 4.8’s thinking returned by the API (left) with our decoding of the thinking block’s signature (Section 2.4). Decoding reveals that the model states the correct answer before attempting to solve the problem.
Figure 9: Divergence of Visible-Answer Style under Opus 4.8 Reasoning Prefill. For each pair of an Opus 4.8 hidden reasoning trace and its visible response, we extract the 1-, 2-, and 3-grams appearing in the visible response. We then measure the number of these n-grams that also appear in outputs generated by Kimi K3 and Inkling under two conditions: (i) the models generate both reasoning and output without intervention (non-prefilled), and (ii) the models’ reasoning is prefilled with the first 1% of tokens from the Opus reasoning trace. The x-axis shows the number of sampled completions, and the y-axis shows the maximum number of shared n-grams within each batch of completions. Curves show the mean over the 15 HLE prompts (32) of each category, STEM and non-STEM (30 in total); shaded bands are ±1 standard error of the mean over the prompts. We observe that Kimi K3’s output changes substantially under the 1% reasoning prefill relative to the non-prefilled control, while Inkling’s does not; Table 3 reports the per-category means and significance tests.
Figure 9: Divergence of Visible-Answer Style under Opus 4.8 Reasoning Prefill. For each pair of an Opus 4.8 hidden reasoning trace and its visible response, we extract the 1-, 2-, and 3-grams appearing in the visible response. We then measure the number of these n-grams that also appear in outputs generated by Kimi K3 and Inkling under two conditions: (i) the models generate both reasoning and output without intervention (non-prefilled), and (ii) the models’ reasoning is prefilled with the first 1% of tokens from the Opus reasoning trace. The x-axis shows the number of sampled completions, and the y-axis shows the maximum number of shared n-grams within each batch of completions. Curves show the mean over the 15 HLE prompts (32) of each category, STEM and non-STEM (30 in total); shaded bands are ±1 standard error of the mean over the prompts. We observe that Kimi K3’s output changes substantially under the 1% reasoning prefill relative to the non-prefilled control, while Inkling’s does not; Table 3 reports the per-category means and significance tests.
Table 4: Breakdown of discovered privacy artifacts by category at each stage of the filtering pipeline. Labeler 1: items flagged by the first-pass LLM-as-a-judge labeler (Haiku 4.5). Labeler 2: items subsequently classified as genuine privacy artifacts. Deduplication: distinct values, grouped by category and value. Non-benchmark: artifacts remaining after excluding benchmark sessions, such as PostTrainBench, TerminalBench, and ClawBench. We note that these sessions may contain genuine artifacts introduced by users running the benchmarks, in addition to benchmark-specific synthetic content. Reasoning only: values that appear nowhere else in the raw session and occur exclusively in the model’s reasoning.
CategoryLabeler 1 (Figure 44)Labeler 2 (Figure 45)DeduplicationNon-benchmarkReasoning only
Personal information
Name4,3505411731304
Address83923387365
Email65123272303
Date of birth12224931
Government ID2921710
Payment card9064900
Phone76181040
Credentials
Access token8528430243
API key96690696211
Password1,23533072332
Private key62111170
Technical identifiers
IP address1,76320660
URL14,1925533323
File or repository path31,38037328127924
Internal identifier14,3692717141
Account identifier3,0723121171
Session identifier1,6626531
Other1,0683429235
Total76,7782,19494170464
Figure 10: Control for Figure 9: the same measurement with the prefill source swapped. As in Figure 9, but neither model is prefilled with proprietary reasoning: Kimi-K3 receives the first 1% of an Inkling trace and is scored against Inkling’s visible answer (left), and Inkling receives the first 1% of a Kimi-K3 trace and is scored against Kimi-K3’s visible answer (right). Same 30 HLE problems, curves are the mean over the 15 prompts of each category; shaded bands are ±1 standard error of the mean over the prompts. Neither model separates from its control in either category.
Figure 10: Control for Figure 9: the same measurement with the prefill source swapped. As in Figure 9, but neither model is prefilled with proprietary reasoning: Kimi-K3 receives the first 1% of an Inkling trace and is scored against Inkling’s visible answer (left), and Inkling receives the first 1% of a Kimi-K3 trace and is scored against Kimi-K3’s visible answer (right). Same 30 HLE problems, curves are the mean over the 15 prompts of each category; shaded bands are ±1 standard error of the mean over the prompts. Neither model separates from its control in either category.
Figure 24: Probabilistic extraction (14) of reasoning traces on 30 HLE problems, k=16 tokens, median over problems (faint lines: individual problems). Rows: extraction of reasoning; extraction of the visible answer under increasing context; the scorer’s own trace as control.
Figure 24: Probabilistic extraction (14) of reasoning traces on 30 HLE problems, k=16 tokens, median over problems (faint lines: individual problems). Rows: extraction of reasoning; extraction of the visible answer under increasing context; the scorer’s own trace as control.
Table 5: Which persona fields the hidden reasoning gives up. Fields of the synthetic Alex Green persona used by ClawBench, against what the reasoning of ClawBenchV2Trace agent runs contains — 75 runs for Opus 4.7, 81 for GPT-5.5, on the same tasks. ✓ = recovered; ✗ = never appears.
Persona fieldGPT 5.5Opus 4.7
identityLegal name (Alex Green)✓✓
Street + unit (664 Spadina Ave, 1208)✓✓
City / province / country✓✓
Postal code (M5S 2H7)✓✓
Security-question answer✗✓
Date of birth✓✗
credsSession email✓✓
Password / session token✗✓
Figure 25: As in Figure 24, on 10 AIME 2025 problems, with GPT-5.6 Sol decoded traces as a second prefill. Kimi-K3 reaches GPT-5.6 Sol’s answer wording within ∼102 queries even conditioned on its own reasoning (third row, right), while no other model does so within 108 queries.
Figure 25: As in Figure 24, on 10 AIME 2025 problems, with GPT-5.6 Sol decoded traces as a second prefill. Kimi-K3 reaches GPT-5.6 Sol’s answer wording within ∼102 queries even conditioned on its own reasoning (third row, right), while no other model does so within 108 queries.
Figure 26: Median perplexity of reasoning traces under seven scoring models. Every model scores perplexity of the reasoning, conditional on the problem, over the same 120 Codeforces problems. Rows are the model whose reasoning is being scored; columns are the model doing the scoring. The diagonal, where a model scores its own reasoning, is set in italic. Colour is on a logarithmic scale, dark for reasoning the scorer finds native.
Figure 26: Median perplexity of reasoning traces under seven scoring models. Every model scores perplexity of the reasoning, conditional on the problem, over the same 120 Codeforces problems. Rows are the model whose reasoning is being scored; columns are the model doing the scoring. The diagonal, where a model scores its own reasoning, is set in italic. Colour is on a logarithmic scale, dark for reasoning the scorer finds native.

Findings

  • The extraction attack successfully recovered plaintext reasoning across all three tested providers (Anthropic, OpenAI, Google) by feeding a strong model's encrypted trace into a weaker sibling (Haiku 4.5, GPT-5.6 Luna, Gemini Robotics 1.6).
  • On 120 Codeforces problems, the API-reported thinking-token counts closely matched the token counts of the re-encoded extracted reasoning for most inputs, supporting the faithfulness of the extraction.
  • Decoding 315,320 reasoning blocks from 6,708 publicly scraped agent sessions recovered 367 PII artifacts and 182 credentials overall, including (from genuine, non-benchmark sessions) 62 API keys, 33 passwords, and 30 personal emails.
  • 64 of 704 artifacts recovered from genuine-session reasoning were entirely absent from the visible chat history, meaning they existed only inside the encrypted block.
  • Decoding Opus 4.8's reasoning after a paraphrased HarmBench prompt revealed harmful information (e.g., car-theft methods) that was completely absent from its benign, refusing visible answer.

Where it can be used

  • Developers using reasoning-model APIs can use this finding as a reason not to publish raw session logs containing encrypted reasoning blocks online without extra caution.
  • AI providers designing or auditing their encryption schemes for reasoning traces can reference this vulnerability analysis when deciding whether to bind traces to specific sessions, users, or models.
  • Security researchers auditing other multi-model API ecosystems can adapt this cross-model compatibility test as a methodology for checking similar flaws.

Limits and open work

  • As of August 2026, providers implemented mitigations following the disclosure, so the exact attacks described are no longer reproducible as-is.
  • The experiments cover a specific set of models and providers (Anthropic, OpenAI, Google); whether the same vulnerability exists in other providers or future model generations is not established.
  • Only 0.3% of the 315,320 decoded blocks actually contained personal data, so the risk should not be generalized to every public reasoning log.
  • Faithfulness of extraction was verified indirectly via matching token counts, not by confirming the extracted content is word-for-word identical to the model's true private reasoning.
  • The cryptographic and system-level mitigations the paper proposes are recommendations, not measures that have been implemented and verified to work.

Why it matters

It shows that filtering only the final visible output of an AI is not enough, because sensitive or harmful content can survive untouched inside the hidden reasoning that gets passed back and forth as an encrypted block. This matters directly for developers who publish session logs online, since they may be unknowingly leaking secrets buried in reasoning they cannot even read.

Terms in this paper

  • chain-of-thought (reasoning trace) · the internal, step-by-step text a model generates to work through a problem before producing its final answer
  • encrypted reasoning block · a scrambled, unreadable chunk of text that packages a model's reasoning trace when it is sent back to the client
  • decoder model · a weaker, less-guarded model that is tricked into reading out another model's encrypted reasoning in plaintext
  • AEAD (authenticated encryption) · an encryption scheme that also proves the content has not been tampered with, using a signature/tag
  • distillation attack · using another model's outputs or reasoning as training data to build a cheaper copycat model

Original abstract (English)

Leading large language model providers now conceal their models' step-by-step reasoning, or chain-of-thought, to protect intellectual property and limit information leakage. Rather than storing these traces server-side, providers return them to the client as blocks of encrypted text, which the client passes back with each subsequent request. Building on prior research, we identify an architectural vulnerability: these encrypted blocks are fully compatible and interchangeable across different sessions, users, and models within a provider's ecosystem. We exploit this compatibility to develop a scalable decryption jailbreak. By injecting an encrypted reasoning trace from a given model into a weaker, and less safeguarded model from the same provider, we force it to decode and output the trace verbatim in plaintext, without ever jailbreaking the more capable model directly. This vulnerability enables four distinct attack vectors. First, it circumvents anti-distillation mechanisms, allowing adversaries to extract a proprietary model's reasoning, as we demonstrate across Anthropic, OpenAI, and Google. Second, it allows for large-scale private data extraction. Developers frequently share session logs publicly, unaware of contents of the encrypted blocks. By decoding 315,320 reasoning blocks scraped from public repositories, we recovered 367 Personally Identifiable Information (PII) artifacts and 182 credentials. Third, it inadvertently reveals hazardous information hidden within the reasoning process, even in cases where the model's final, visible output safely rejects a malicious request. Fourth, attackers can leverage this flaw to execute invisible prompt injections, embedding malicious payloads entirely within encrypted blocks to poison public agentic rollouts. Following responsible disclosure, we propose concrete cryptographic and system-level mitigations to secure client-side reasoning.

Authors · Alexander Panfilov

Read on arXiv

Latest papers

All papers →

Latest from METAL LAB

Figures: Alexander Panfilov et al., arXiv:2608.09867, CC BY 4.0