SHAPE of Chain-of-Thought in Math Reasoning

arXiv:2608.286002026-09-01

给AI解数学题时的每一步思考打上标签,看它用了什么策略、卡在哪种理解方式里,由此揭示答案对错的原因以及强化学习训练带来的副作用

SHAPE用数学教育研究中的两个概念来分析大模型的思维链(CoT):语义空间(模型当前对题目的数学理解方式,比如代数式还是几何式)和启发式(具体的解题动作,比如逆向推理或化简问题)。通过自动化标注流程测量后发现,基于启发式的特征比CoT长度等表面特征更能预测答案是否正确,而强化学习后训练出的模型反而把策略收缩到基础模型已有的窄范围内,把启发式信息直接加入训练(HA-Plan-GRPO)则能提升准确率。

METAL LAB 解读图

SHAPE如何把思维链转化为可测量的结构

证据状态实测结果与计划中的工作并存

原始思维链模型解数学题时逐步写出的解题文字过程
切分为内容单元把文本切成能代表一个解题动作的最小片段
标注启发式给每个单元打上逆向推理、引入表示法等具体数学动作标签
追踪语义空间判断每个单元相对模型不断演变的题目理解方式是开启新空间(New)、回到旧空间(Return)还是维持当前空间(Maintain)
计算指标并应用于训练提取能预测答案正确性的特征,测量后训练带来的Density与Coverage收窄现象,并把启发式重新注入GRPO的rollout提示中

这是 METAL LAB 制作的解读图，并非论文作者提供的原图。

他们做了什么

SHAPE用两个源自数学教育研究的概念分析思维链文本:语义空间指模型当前对题目采用的数学理解方式(如代数式或几何式),启发式指逆向推理、引入符号、验证特例等具体解题动作。
一套用Grok-4.1-Fast、Qwen3.5-27B等模型搭建的自动化流程把思维链切成小的内容单元,给每个单元标注启发式标签,并判断每个单元是开启新的语义空间、回到旧的语义空间,还是维持当前语义空间;该流程在48条人工标注的黄金样本上得到验证。
用SHAPE的启发式频率特征训练的逻辑回归在预测答案正确性上达到AUROC 0.664,超过CoT长度(0.504)、长度加推理token特征(0.503)、诸如wait/aha等自我修正词标记(0.618)以及现有的情节标签方法ThinkARM(0.618)。
用Density和Coverage指标比较基础模型与经强化学习后训练(Think-RLVR)模型的启发式使用模式,发现所有被测试的后训练模型对中Density均超过1.0、Coverage均低于1.0,说明后训练并未产生新策略,而是把成功的解题路径收缩进基础模型已有策略中较密集的一小块区域;作为对照,两个互不相关模型之间的Density(0.520)和Coverage(0.437)都很低,证实这一现象只出现在基础模型与其后训练版本之间。
在用GRPO对Qwen3-1.7B-Base做MATH题训练的实验中,在rollout提示中显式加入启发式信息的HA-Plan-GRPO,在MATH-Perturb测试集上的Avg@64和Pass@64均高于只加规划步骤但不含启发式信息的Plan-GRPO(该结果被作者标注为初步结果)。

Figure 1: Illustrative example of SHAPE annotation applied to a synthetic CoT trace. The solver first sets up an algebraic formulation (Space 1), introducing a system of equations. It then abandons this approach and switches to a trial-based counting strategy (Space 2). Finally, it returns to the algebraic formulation (Space 1) and solves the system. We refer to each problem-solving interpretation as a semantic space—a distinct mathematical interpretation defined by the objects, goals, and constraints the model adopts. Within each space, individual steps are tagged as heuristics—purposeful mathematical actions such as introducing a representation (H3a) or exploring particular cases (H9a)—or as non-heuristic steps such as stating the answer (N4). The arrows mark transitions between these spaces. For the full heuristic taxonomy, see Appendix A.

Table 1: Correctness prediction under 5-fold stratified cross-validation. “Length + reasoning” includes CoT length together with reasoning features (reasoning-token count and proportion), while “Self-revision” includes self-revision features derived from lexical self-revision markers. Within SHAPE, H denotes the 11 heuristic categories (H1–H11) and N the non-heuristic category.
Feature Set	AUROC ↑	# Features
Length	0.504±0.03	1
Length + reasoning	0.503±0.03	3
Self-revision	0.618±0.03	3
ThinkARM	0.618±0.02	8
SHAPE (H)	0.653±0.02	11
SHAPE (H+N)	0.664±0.02	12

Figure 2: Overview of the automated SHAPE analysis pipeline. Given a Chain-of-Thought (CoT) trajectory, the pipeline first segments the text into content units corresponding to heuristic-bearing spans. It then assigns one or more heuristic labels to each unit using a tagging model. Finally, a semantic-space state tracking model classify the model’s mathematical interpretation change into Maintain, New, or Return.

Table 2: Descriptive statistics of SHAPE metrics by correctness. Nspaceeff: effective number of semantic spaces; Ntranseff: effective number of semantic-space transitions; transition ratio ρ: measuring the intensity of revisiting semantic space. (C), (I), and (O) denote correct, incorrect, and overall trajectories.
	Nspaceeff	Ntranseff	ρ
Model	(C)	(I)	(O)	(C)	(I)	(O)	(C)	(I)	(O)	Acc.
Open-source reasoning models with full traces
Qwen3-32B	2.08	2.28	2.16	1.38	1.53	1.44	0.51	0.50	0.51	.63
DeepSeek-R1	1.72	2.47	2.00	0.82	1.64	1.12	0.34	0.51	0.41	.63
QwQ-32B	1.74	2.74	2.11	0.86	2.20	1.35	0.32	0.60	0.42	.63
DeepSeek-R1-Distill-Qwen-32B	1.85	2.07	1.95	1.02	1.21	1.10	0.38	0.41	0.40	.57
DeepSeek-R1-Distill-Qwen-7B	1.51	2.18	1.85	0.60	1.40	1.01	0.27	0.49	0.41	.49
Deepseek-R1-Distill-Qwen-1.5B	1.41	2.04	1.81	0.53	1.27	1.00	0.27	0.49	0.41	.36
Phi-4-Reasoning	2.03	2.83	2.53	1.15	2.10	1.75	0.35	0.58	0.49	.35
Instruction-tuned models without extended reasoning
Qwen3-32B-NR	1.66	1.74	1.71	0.70	0.75	0.73	0.29	0.33	0.32	.34
Gemini-2.0-Flash	1.43	1.58	1.53	0.43	0.59	0.54	0.20	0.28	0.25	.34
Phi-4	1.14	1.51	1.41	0.14	0.51	0.41	0.07	0.23	0.19	.29
Qwen-2.5-32B	1.33	1.39	1.37	0.37	0.42	0.40	0.16	0.20	0.19	.28
GPT-4o	1.68	1.40	1.46	0.72	0.40	0.46	0.29	0.19	0.21	.19
Proprietary reasoning models with hidden traces
Gemini-2.5-Flash	1.56	1.91	1.67	0.59	0.91	0.69	0.28	0.35	0.30	.67
GPT-o3-mini	1.51	1.65	1.59	0.56	0.70	0.63	0.27	0.32	0.30	.47
GPT-o1-mini	1.36	1.58	1.50	0.36	0.61	0.52	0.19	0.29	0.25	.37

Figure 3: Projection of successful Olmo-3-{Base, Think-RLVR} trajectories onto the first principal component of heuristic frequency space. Olmo-3-Think-RLVR concentrates around the peak of the base distribution while leaving the left tail uncovered, suggesting that post-training narrows rather than reshapes the heuristic distribution.

Table 4: Density and Coverage of post-trained model trajectories relative to base model trajectories in heuristic frequency space (k=3, successful trajectories only, aggregated across all perturbation conditions). Density > 1 indicates that post-trained trajectories concentrate in the dense core of the base distribution; among such cases, lower Coverage indicates stronger mode-seeking. The cross-model baseline confirms that unrelated base models show neither high Density nor high Coverage.
Base	Post-trained	Nbase	NPT	Density	Coverage
Post-trained
Qwen3-1.7B-Base	Qwen3-1.7B-GRPO	834	886	1.220	0.871
Olmo-3-7B	Olmo-3-7B-Think-RL-Zero	1229	1307	1.250	0.707
Olmo-3-7B	Olmo-3-7B-Think-RLVR	1507	1600	1.032	0.531
Cross-model baseline (unrelated distributions)
Olmo-3-7B	Qwen3-1.7B-Base	71	66	0.520	0.437

Table 5: Preliminary results for Heuristic-Augmented GRPO on the MATH-Perturb test split. Plan-GRPO adds a planning step without heuristic information. HA-Plan-GRPO provides eleven mathematical heuristics during rollout and incorporates them into a planning-guided rollout. Bold indicates the best results.
Model	Original	Simple	Hard
Avg@64	Pass@64	Avg@64	Pass@64	Avg@64	Pass@64
Qwen3-1.7B-Base	23.54	77.40	23.10	79.13	11.84	57.39
+ Plan-GRPO	30.00	80.00	29.86	78.26	14.52	61.74
+ HA-Plan-GRPO (ours)	36.80	80.00	35.80	79.13	17.72	62.61

Table 6: Taxonomy of Problem-Solving Strategies. This table summarizes the code, strategy name, detailed description, and theoretical sources associated with each heuristic. Here, P = Pólya (1945), K = Koichu et al. (2007), F = Favier (2022), and P&K = Posamentier and Krulik (2008).
Strategy Name	Description & Sources
H1. Changing the register of semiotic representation	This strategy involves translating the problem’s representation from one semiotic register to another. It includes converting between natural language, algebraic, geometric, and visual representations to facilitate understanding or solving. Sources: Creating a model (K); Change the semiotic representation register (Changer de registre de représentation sémiotique) (F)
H2. Cognitive Reinterpretation	This involves changing the way an object or property in the problem is interpreted. It redefines the identity or attributes of an element in a way different from the initial presentation, without necessarily changing the register. Sources: Consider another way of interpreting the problem’s objects (Envisager une autre façon d’interpréter les objets du problème) (F); Adopting a different point of view (P&K)
H3. Introduce Symbolic Representation, Formalization, and Structural Augmentation, including…
H3a. Introduce Symbolic Representation and Formalization	The act of introducing new variables, labeling unknowns, or performing substitutions to make ambiguous targets operationally manageable. Sources: Notation; Setting up equations (P); Creating a model (K); Introduce names or notations (Introduire des noms ou des notations) (F)
H3b. Structural Augmentation	Constructing auxiliary objects, lemmas, or entirely new mathematical frameworks that are not present in the original problem. This is a creative addition to the problem space, such as drawing auxiliary lines, defining new functions, or shifting the problem into a new structural representation Sources: Auxiliary elements, Lemma (P); Introducing an auxiliary element (K); Introduce auxiliary elements (Introduire des éléments auxiliaires) (F)
H4. Problem Classification / Rephrase the Problem and Goal, including…
H4a. Problem Categorization / Strategic Rephrasing of Goal / Breaking into Sub-goals	Explicitly stating the problem type, identifying applicable solution methods, or reformulating the main goal in clearer mathematical terms. Sources:What is the unknown? (P); Reformuler le problème (Reformulate the problem) (F)
H4b. Filtering Constraints	Strategically identifying the most essential constraints or conditions that guide the upcoming solution approach. Sources:Separate the various parts of the condition (P); Exploring a particular datum (K)
H5. Wishful Thinking (Simplify / Reduce the Problem and Conditions)	Temporarily modifying the problem to a simpler version to gain insight, verify formulas, or explore solution strategies. Sources: If you cannot solve the proposed problem (P); Reduce the problem to a simpler one (Réduire le problème à un problème plus simple) (F)
H6. Explicit Case Analysis, Decompose into Subproblems	Logically decomposing the problem into distinct cases, non-overlapping subsets, or sub-problems that, when combined, yield the full solution. The cases should ideally be exhaustive and mutually exclusive. Sources: Decomposing and recombining (P); Décomposer le domaine du problème et travailler cas par cas (Decompose the problem domain and work case by case) (F)
H7. Arguing by contradiction	A proof strategy where the negation of the proposition is assumed to derive a contradiction, thereby proving the original statement. Sources: Reductio ad absurdum and indirect proof (P); Arguing by contradiction (K)
H8. Analogy and Presenting Related Theorems, including…
H8a. Analogy	Recalling previously solved problems, known methods, or applying a recently established logical procedure to a new target within the same problem. This involves recognizing structural similarities and transferring a strategy from one context (or one part of the equation) to another. Sources: Have you seen it before?; Do you know a related problem? (P); Activating a previous experience (K); Make a connection with a previously encountered problem (Faire le lien avec un problème déjà rencontré) (F); Solving a simpler analogous problem (P&K)
H8b. Presenting Related Theorems, Tools, or Properties	Introducing specific mathematical theorems, formulas, identities, or properties that are not provided in the problem statement but are necessary to advance the solution. Sources: Connect with a mathematical tool (theorem, property) (Faire le lien avec un outil mathématique (théorème, propriété)) (F)
H9. Experimental and Pattern Exploration, including…
H9a. Exploring particular cases or numbers	Plugging in specific values, extreme/boundary values, or limits to discover patterns, build intuition, or verify feasibility. Sources: Specialization (P); Partial Induction (K); Explore a specific piece of data (Explorer une donnée particulière) (F); Finding a Pattern (P&K)
H9b. Exploration of symmetry	Identifying and exploiting mathematical or structural symmetry to reduce the solution space or simplify computation. Sources: Symmetry (P); Exploration of Symmetry (K); Exploit symmetry properties (Exploiter les propriétés de symétrie) (F)
H10. Thinking from the end to the beginning (Working backward)	Starting from the desired conclusion (target goal) and working logical steps backward to reach the known premises or to determine what would be sufficient to prove. Sources: Working backwards (P); Thinking Backward (K); Working backward (Travailler à reculons) (F); Working Backwards (P&K)
H11. Verification and Looking Back, including…
H11a. Re-solving & Checking the Argument	Re-performing the same logical steps or calculations without a strategic change, or conducting a direct manual check of elements to verify a previous claim. Sources: Can you check the result? (P); Local Self-evaluating (K)
H11b. Deriving the Result Differently	Solving the same problem or sub-goal using a structurally different mathematical method to provide independent confirmation. Sources: Can you derive the result differently? (P); Local Self-evaluating (K)
H11c. Backtracking for self-verification	Realizing an error, finding a flaw in an assumption, or recognizing that the current approach is not working, and revising the direction. Sources: Backtracking (Retour arrière) (F)
H11d. Checking the Result / Sanity Check / Progress Review	Broadly covers any reflection on whether the solution is on the right track or checking feasibility. Sources: Can you check the result?; Test by dimension (P); Local Self-evaluating (K)
H11e. Generalization & Corollary	Extending the result to broader cases. Identifying general principles from specific solutions. Sources: Wisdom of proverbs (P); Generalization (K)
H11f. Reflect on Rigor & Wisdom	Evaluating the efficiency of the solution strategy, questioning the rigor, or meta-reflecting on definitions/rules. Sources: Why proofs? (P)

Table 8: Heuristic-tagging performance of candidate annotator models on the gold set. Weighted F1 reflects overall label agreement, while macro F1 emphasizes performance on rare heuristic classes.
Model	Weighted F1	Macro F1
Grok-4.1-Fast	76.98	65.04
GPT-5	72.25	63.55
GPT-5-mini	66.44	55.89
Gemini-3-Flash	62.83	41.79
Gemini-3-Flash-Lite	70.21	54.14
Qwen3.5-27B	70.44	61.36

Table 9: Class-wise heuristic-tagging performance of Qwen3.5-27B on the gold set. We report precision (P), recall (R), F1, Cohen’s kappa, and the number of gold and predicted instances.
Class	F1	Kappa
H1	0.5272	0.4893
H2	0.4318	0.4176
H3	0.4884	0.4383
H4	0.6114	0.5049
H5	0.6250	0.6134
H6	0.6667	0.6626
H7	0.6667	0.6664
H8	0.7201	0.6252
H9	0.5934	0.5692
H10	0.5000	0.4988
H11	0.8115	0.6383
N	0.7216	0.6702

Table 11: Hyperparameters used for GRPO training.
Parameter	Value
Base Model	Qwen/Qwen3-1.7B-Base
Training Batch Size	32
Rollouts per Prompt	4
Number of GPUs	2×B200
Optimizer	AdamW
Temperature	1.0
Top-p	1.0
Top-k	-1
Max Response Length	2048
Learning Rate	1×10−6
Warmup Steps	10
Training Steps	200

研究结果

SHAPE的启发式频率特征在预测答案正确性上达到AUROC 0.664±0.02,高于CoT长度(0.504±0.03)、长度加推理token特征(0.503±0.03)、自我修正词标记(0.618±0.03)以及ThinkARM情节标签方法(0.618±0.02)。
具备扩展思考能力的推理模型在语义空间覆盖度(Nspaceeff 1.81至2.53)和转移比率(rho 0.40至0.51)上均高于不做扩展推理的指令微调模型(Nspaceeff 1.37至1.71,rho 0.19至0.32);在大多数模型中,答错的解题过程比答对的表现出更高的rho值。
在要求换用不同解题思路的高难度扰动题上,模型的启发式频率分布、语义空间数量和转移比率变化都比只保留原解法结构的简单扰动题更大,且这种差异在思维链最初的5到10个内容单元内就已出现。
被测试的所有基础模型与后训练模型对(Olmo-3-7B的Think-RLVR和Think-RL-Zero版本、Qwen3-1.7B-Base搭配GRPO)Density均超过1.0、Coverage均低于1.0;作为对照的两个互不相关模型之间Density仅为0.520、Coverage仅为0.437。
在MATH-Perturb测试集上,Plan-GRPO和HA-Plan-GRPO都比未训练的基础模型表现更好,而在rollout提示中加入启发式信息的HA-Plan-GRPO在Avg@64和Pass@64上都高于Plan-GRPO(该结果为初步结果)。

可应用场景

不只看最终答案,而是通过检查模型用了哪些启发式、如何在语义空间之间切换,来诊断模型在某类题目上失败的原因。
用来检查一次强化学习后训练究竟是让模型的解题策略更加多样,还是只是把它收窄到已经会的那几种解法里。
参考HA-Plan-GRPO的做法,尝试在强化学习训练的rollout提示中显式加入启发式词汇,以期提升数学基准上的表现。
把内容分割、启发式标注、语义空间追踪这套流程的设计思路搬到其他结构化推理任务上,不过目前这一具体实现仅在数学题上得到验证。

局限与待验证事项

这套框架及其启发式分类体系目前只在数学推理基准(MATH-Perturb、Omni-MATH子集)上开发和验证过,扩展到其他领域仅被作者列为未来工作,尚未实际验证。
启发式标注的黄金标准仅来自四个模型产生的48条解题过程,而语义空间追踪本身没有客观的黄金标准,只能通过反复人工审查来校准提示词。
HA-Plan-GRPO的结果被作者明确标注为初步结果,且只在一个基础模型(Qwen3-1.7B-Base)和一种训练方案上测试过,尚未在更大模型、不同模型家族或其他强化学习算法上验证。
像H7、H10这类黄金样本数量很少的稀有启发式类别,标注模型的表现较弱,这可能让某些细粒度结论不够稳定。
自动标注依赖Grok-4.1-Fast、Qwen3.5-27B这类大型或专有模型作为标注器,可能给后续所有分析带来标注噪声或模型特有的偏差。

为什么重要

只看最终答案对不对,很难知道模型到底是怎么想的,也很难判断强化学习是真的提升了数学能力,还是只是让模型反复用已经会的那几招。SHAPE提供了一套有理论依据的方法来观察这个过程,并且用具体实验给出了一个把启发式信息加入训练后确实能提升表现的例子。

本文术语

思维链(Chain-of-Thought, CoT) · 模型在给出最终答案前,一步一步写出来的解题过程文字
语义空间 · 模型当前对题目采取的数学理解方式,比如把题目当作代数问题还是几何问题来处理
启发式 · 逆向推理、化简问题、验证特例等有明确目的的具体解题动作
GRPO · 用于对模型做强化学习微调的方法,全称Group Relative Policy Optimization
Density与Coverage · Density衡量目标分布在参考分布密集区域的聚集程度,Coverage衡量目标分布覆盖了参考分布中多大比例的范围

论文原文摘要（英文）

Large language models (LLMs) achieve strong performance on mathematical reasoning benchmarks, yet the mathematically meaningful skills underlying their reasoning remain underexplored. We introduce \texttt{SHAPE}, a framework that analyzes Chain-of-Thought (CoT) trajectories through two lenses developed in mathematics education: (1) semantic spaces: the model's evolving mathematical interpretations of a problem (e.g., algebraic, geometric), and (2) heuristics: the specific mathematical actions taken within those spaces (e.g., simplifying the problem, working backward). We first use \texttt{SHAPE} to analyze the reasoning patterns of various models. Our findings reveal that the mathematical heuristics employed by a model better explain final answer correctness than traditional CoT features. Furthermore, models are likely to reach correct solutions by concentrating their reasoning effort within a few semantic spaces rather than exploring many disparate ones -- a pattern consistent with human behavior. Next, we utilize the \texttt{SHAPE} lens to evaluate whether post-training truly enhances mathematical proficiency. We find that reinforcement learning induces mode-seeking in heuristic usage. Lastly, we post-train LLMs by promoting diverse heuristics and demonstrate its effectiveness in improving accuracy. Overall, \texttt{SHAPE} provides a theoretically-grounded diagnostic framework for decoding LLM reasoning and offers a new path toward post-training LLMs for math reasoning. The code for our model is available at https://github.com/holi-lab/SHAPE-of-CoT

作者 · Jonghyun Song, Sangjun Song, Minjae Oh, Haesung Pyun, Sungsik Lee, Yohan Jo

在 arXiv 阅读

METAL LAB 最新报道

图片来源： Jonghyun Song et al., arXiv:2608.28600, CC BY 4.0