이미지: METAL LAB 생성
Summary
- A paper reportedly presents a method to recover 100% of the hidden reasoning tokens from Claude and GPT model families, according to the community
- Exposed reasoning traces allegedly show Claude responding to AIME problems as if it had memorized them in advance
- The post also observed that the "weird word strings and overthinking" pattern commonly seen in open-source models also appears in frontier models
- 논문 제목
- Stealing Reasoning Traces from Proprietary LLM APIs (커뮤니티 전언)
- 대상 모델
- Claude 계열, GPT 계열 전체로 알려짐
- 핵심 주장
- 비공개 API 추론 토큰 100% 복원 가능
- 논란 지점
- Claude가 AIME 벤치마크 문제를 암기한 듯한 반응 확인 (커뮤니티 주장)
- 확산 경로
- r/LocalLLaMA 게시글, 2026년 8월 12일
What happened
Claude and GPT-series models offered via API by OpenAI and Anthropic don't fully expose "reasoning tokens" — the internal thinking process a model goes through before producing an answer — to users. The standard practice has been to show only a summarized version, or to hide it entirely. But a Reddit user in r/LocalLLaMA reported on August 12 that a paper had found a way to break through this barrier and recover 100% of the reasoning tokens from both companies' model families. The paper reportedly included numerous examples of actual recovered reasoning traces.
The most notable part of the post concerns a benchmark-related suspicion. When AIME benchmark questions — drawn from a math competition problem set — were given to Claude, the recovered reasoning process allegedly showed signs that the model already "knew" the answer, as if it had memorized it. If true, this suggests that some of the performance charts showing Claude outperforming open-source models may be inflated beyond their actual capability.
What this means
This isn't the first time such controversy has surfaced. Most AI performance metrics are measured using publicly available benchmark problem sets, and if the questions and answers have already found their way into training data, then a model isn't "solving" the problem — it's "recalling" it. This is often called "benchmaxing": optimizing training toward specific problem patterns just to produce good benchmark scores. Epoch AI's release of a new, non-public game-puzzle benchmark on August 6 was driven by the same concern — an attempt to measure genuine reasoning ability using problems models were unlikely to have already learned through post-training. At the time, the top score recorded was 59%, achieved by Opus 5.
The post added another interesting observation. The "weird word strings" or "overthinking" behavior often seen when running open-source models is, it argues, actually a normal pattern that appears almost universally in frontier models too. Such behavior had long been treated as evidence that open-source models were less refined, but once the actual reasoning processes of closed models were examined, they turned out not to be so different.
What changes now
Based on this discovery, the post's author argued that open-source models aren't actually as far behind as commonly believed. The point, in short: "there's no secret sauce — it's just data, compute, and engineering." Some in the community speculated that Chinese companies may have been exploiting this information gap for frontier-model distillation — the process of transferring a large model's reasoning patterns to train a smaller one. With this barrier now reportedly closed, some expect such distillation efforts to slow down.
However, all of this information comes not from the paper itself but from a Reddit post describing it, so the paper's exact methodology, recovery accuracy, and whether the two companies have responded all still require further confirmation.
