Recursive Agentic Reasoning

arXiv:2608.239562026-08-26

Sampling several answers and voting on them also fixes cases where an AI simply fails to produce any answer at all

Researchers compared three ways of giving language models more thinking time at inference — deepening one reasoning path (Grow), decomposing a problem into sub-questions (Prune), and sampling multiple answers and voting (Branch) — under identical conditions. Across 14 model-benchmark combinations spanning 3 frontier models and 5 benchmarks, Branch improved accuracy most consistently. Digging into why, they found Branch wins largely because it recovers cases where the model's response got cut off before producing any answer, not just because it combines multiple reasoning paths.

METAL LAB explanatory visual

Comparing three test-time reasoning operators

Evidence statusMeasured results reported

CoT baselineThe model is called once and returns whatever single answer it produces
GROWFeeds the previous answer back in and re-solves, stopping once the answer stops changing (up to 3 rounds)
PRUNEDecomposes the problem into ordered sub-questions, solves them in sequence, then recomposes an answer
BRANCHDraws 5 independent answers and picks the majority; empty responses are automatically excluded from the vote
Key findingBRANCH wins largely by recovering cases where the baseline ran out of tokens and returned nothing (r=0.72 correlation with baseline empty-output rate)

An explanatory diagram made by METAL LAB, not a figure supplied by the paper's authors.

What they did

The authors unify three test-time reasoning methods into one framework of 'recursion operators': GROW (repeatedly deepen a single reasoning path), PRUNE (decompose a problem into sub-questions, solve them, then recompose), and BRANCH (sample multiple independent answers and pick the majority).
They compared all three operators against a single-pass chain-of-thought (CoT) baseline using identical prompts, token budgets, and grading code in one shared test harness, across 3 frontier models (DeepSeek-V4-Pro, MiniMax-M3, Qwen3.6-plus) and 5 benchmarks, totaling 14 model-benchmark cells, 49,327 graded items, and 151,876 model calls.
BRANCH improved accuracy in all 14 cells by an average of 5.98 percentage points over the CoT baseline and was the best-performing operator in 12 of them. GROW averaged only +2.18 points and actually hurt performance in two cells, while PRUNE averaged just +0.94 points.
Analysis showed BRANCH's advantage comes not only from combining multiple reasoning paths but also from recovering answers when the baseline ran out of its token budget mid-reasoning and returned nothing at all: BRANCH's gains correlated strongly (r=0.72) with how often the baseline produced empty, budget-exhausted outputs.
The paper also demonstrated that comparing methods on unpaired (mismatched) item sets, or treating scoring-infrastructure failures as model errors, can materially change or even reverse comparative conclusions — and proposes paired scoring, where every method is graded on exactly the same resolved items, as standard practice.

Figure 1: The four inference-time computation graphs compared in this paper. Hollow nodes are the problem, filled nodes are model calls, and the bottom node is the returned answer. CoT issues one call. Grow extends a single path and halts when the extracted answer stops changing between rounds. Prune decomposes into ordered sub-questions, solves each with the previous answers in context, and recomposes. Branch samples N=5 independent solutions and selects by majority vote over normalized answers, which also discards samples that returned nothing. Call counts are means measured over all runs.

Table 1: Benchmarks. n is the number of items evaluated per operator per model. MuSiQue supplies all supporting and distractor paragraphs in context, so it measures multi-hop reasoning rather than retrieval. HLE is restricted to its text-only subset. †BBEH replaces each task of BIG-Bench Hard (Suzgun et al., 2023) with a harder counterpart probing the same skill.
Benchmark	Capability probed	n	Metric
MuSiQue (Trivedi et al., 2022)	Multi-hop composition	2,417	EM / F1
HLE (Phan et al., 2026)	Expert-level academic	2,158	Normalized EM
BBEH (Kazemi et al., 2025)	General many-hop†	200	Normalized EM
SuperGPQA (M-A-P Team et al., 2025)	Graduate knowledge	300	Multiple choice
Omni-MATH (Gao et al., 2025)	Olympiad mathematics	300	Normalized EM

Figure 2: Accuracy change relative to the single-pass CoT baseline in each of the 14 model × benchmark cells, sorted by Branch gain. Bars right of zero are improvements. Branch is positive in every cell; Grow and Prune each go negative in two. All three operators are scored on the same paired item set as the baseline within a cell.

Table 2: Accuracy (%) by benchmark, model, and operator, with the change from the CoT baseline in parentheses. n is the paired item count (Section 4.2). Bold marks the best operator in each cell. Omni-MATH was not run on Qwen3.6-plus, leaving 14 of 15 possible cells.
Benchmark	Model	n	CoT	Grow	Prune	Branch
MuSiQue	DeepSeek-V4-Pro	2,417	67.94	66.69 (−1.25)	68.43 (+0.49)	72.36 (+4.42)
MuSiQue	MiniMax-M3	2,398	73.35	75.23 (+1.88)	74.23 (+0.88)	75.85 (+2.50)
MuSiQue	Qwen3.6-plus	2,395	77.08	77.49 (+0.41)	74.91 (−2.17)	78.46 (+1.38)
HLE	DeepSeek-V4-Pro	2,152	13.34	14.82 (+1.48)	14.68 (+1.34)	19.93 (+6.59)
HLE	MiniMax-M3	500	22.40	27.80 (+5.40)	23.80 (+1.40)	31.00 (+8.60)
HLE	Qwen3.6-plus	305	16.39	19.34 (+2.95)	15.08 (−1.31)	17.05 (+0.66)
BBEH	DeepSeek-V4-Pro	200	38.50	36.00 (−2.50)	37.50 (−1.00)	52.00 (+13.50)
BBEH	MiniMax-M3	200	26.50	32.50 (+6.00)	30.50 (+4.00)	37.00 (+10.50)
BBEH	Qwen3.6-plus	192	66.67	68.75 (+2.08)	68.75 (+2.08)	72.40 (+5.73)
SuperGPQA	DeepSeek-V4-Pro	300	58.33	59.00 (+0.67)	60.67 (+2.34)	65.33 (+7.00)
SuperGPQA	MiniMax-M3	299	58.53	62.54 (+4.01)	59.20 (+0.67)	65.22 (+6.69)
SuperGPQA	Qwen3.6-plus	300	69.67	72.00 (+2.33)	69.67 (+0.00)	72.00 (+2.33)
Omni-MATH	DeepSeek-V4-Pro	299	33.11	33.11 (+0.00)	35.79 (+2.68)	39.46 (+6.35)
Omni-MATH	MiniMax-M3	282	30.14	37.23 (+7.09)	31.91 (+1.77)	37.59 (+7.45)

Figure 3: (a) Per-cell Branch gain against the baseline’s empty-output rate, with a least-squares fit; the association is strong (r=0.72, n=14). The four cells at zero on the horizontal axis are the Qwen3.6-plus cells, which never truncated, and they show the smallest gains. (b) Empty-output rate under the baseline and under Branch for the ten cells where truncation occurs at all. Branch roughly halves it everywhere.

Table 3: Attempted versus paired item counts. Six of the fourteen cells are unaffected. The HLE Qwen3.6-plus cell is the outlier and is the cell discussed in Section 6.
Benchmark	Model	Attempted	Paired n	Attrition
MuSiQue	DeepSeek-V4-Pro	2,417	2,417	0
MuSiQue	MiniMax-M3	2,417	2,398	19
MuSiQue	Qwen3.6-plus	2,417	2,395	22
HLE	DeepSeek-V4-Pro	2,158	2,152	6
HLE	MiniMax-M3	500	500	0
HLE	Qwen3.6-plus	500	305	195
BBEH	DeepSeek-V4-Pro	200	200	0
BBEH	MiniMax-M3	200	200	0
BBEH	Qwen3.6-plus	200	192	8
SuperGPQA	DeepSeek-V4-Pro	300	300	0
SuperGPQA	MiniMax-M3	300	299	1
SuperGPQA	Qwen3.6-plus	300	300	0
Omni-MATH	DeepSeek-V4-Pro	300	299	1
Omni-MATH	MiniMax-M3	300	282	18

Figure 4: Mean accuracy gain against mean model calls per item, averaged over all 14 cells. Dotted guides mark constant returns per call. Branch buys the most accuracy; Grow is marginally more efficient per call but has a far lower ceiling; Prune is dominated on both axes.

Findings

BRANCH beat the CoT baseline in all 14 model-benchmark cells, with a mean gain of 5.98 percentage points (median 6.47), and was the strict best operator in 12 of them.
GROW averaged +2.18 points but went negative on DeepSeek-V4-Pro for MuSiQue (-1.25) and BBEH (-2.50); PRUNE averaged only +0.94 points.
Under the baseline, DeepSeek-V4-Pro returned empty output on 51.2% of HLE items, 34.5% of BBEH items, and 36.1% of Omni-MATH items due to token-budget exhaustion; BRANCH roughly halved these empty-output rates (e.g., HLE from 51.2% to 32.2%).
BRANCH's per-cell accuracy gain correlated strongly with the baseline's empty-output rate (r=0.72), and Qwen3.6-plus, the only model that never truncated, showed the smallest BRANCH gains.
An earlier unpaired analysis appeared to show Qwen3.6-plus's HLE accuracy dropping from 14.20% (baseline) to 11.20% under BRANCH, but this was an artifact of scoring unresolved items (lost to network timeouts) as wrong; on the paired item set, BRANCH actually improved accuracy by +0.66 points.

Figure 5: HLE accuracy on the 305 items resolved by every model × operator run. Restricting to a shared item set changes the model ranking relative to the unpaired numbers in Table 2, because the 500-item subset attempted by two of the models is easier than the full split.

Where it can be used

When facing a fixed extra compute budget to improve an AI model's accuracy on hard problems, sampling multiple answers and majority-voting is a reasonable first method to try.
For long-reasoning models that frequently hit their token budget and return nothing, a cheap finalization pass that asks the model to commit to an answer from its truncated reasoning may recover much of the benefit of full repeated sampling at lower cost — worth investigating.
When designing experiments comparing multiple test-time reasoning methods, adopting paired scoring — evaluating all methods only on the item set every method actually resolved — helps avoid misleading comparisons.
Grading pipelines should exclude items lost to transport/API failures rather than automatically marking them wrong, to avoid unfairly penalizing methods that make more model calls.

Limits and open work

BRANCH here uses only unweighted majority voting with no verifier or confidence weighting, which the authors note is less expressive than verifier-guided selection methods.
BRANCH is flat parallel sampling rather than tree search — it does not expand or prune partial reasoning states the way methods like Tree of Thoughts do.
No formal significance testing was reported; at roughly 200 items per cell, the 95% confidence interval is about ±7 points, so small effects on BBEH and Omni-MATH should be treated as provisional.
The HLE grader is a conservative lower bound on true accuracy, and Omni-MATH was never run on Qwen3.6-plus.
All three tested models are reasoning models with hidden deliberation streams, so the truncation-recovery mechanism identified may not transfer to models without that behavior.

Why it matters

For practitioners deciding how to spend a limited inference-time compute budget, this gives concrete evidence about which method to reach for and why. It's also a cautionary methodological finding: how you handle grading and pairing can flip your conclusions about which method actually works best.

Terms in this paper

test-time reasoning · Techniques that boost a model's accuracy at answer time by spending extra computation, without retraining the model
GROW / PRUNE / BRANCH · The paper's three defined operators: repeatedly refine one path / decompose into sub-questions / sample many answers and vote
chain-of-thought (CoT) baseline · The simple reference method where the model is called just once and produces its answer through one reasoning trace
paired scoring · Comparing methods only on the exact same set of items that every method successfully answered
truncation / budget-exhausted output · When a model uses up its allotted token budget while still 'thinking' internally and never reaches a final answer, resulting in an empty response

Original abstract (English)

Test-time reasoning methods such as iterative refinement, decomposition, and repeated sampling are often evaluated in isolation, making their gains difficult to compare across models, benchmarks, and evaluation pipelines. We introduce a unified view of these methods as recursion operators over an agent's reasoning trace: GROW, which deepens a single reasoning path; PRUNE, which decomposes and recomposes the problem; and BRANCH, which samples alternative reasoning paths and selects among them. We evaluate all three operators against a single-pass chain-of-thought baseline under a shared harness with identical prompts, token budgets, and grading code. Across five benchmarks and three frontier models, comprising 14 model-benchmark settings, 49,327 graded items, and 151,876 model calls, BRANCH improves accuracy in all 14 settings by an average of 5.98 percentage points and is the best-performing operator in 12. In contrast, GROW yields a mean gain of 2.18 points and degrades performance in two settings, while PRUNE improves accuracy by 0.94 points on average. Analysis shows that BRANCH's advantage arises not only from exploring multiple reasoning paths, but also from recovering from truncation: its gains strongly correlate with the baseline rate of empty, budget-exhausted outputs (r = 0.72). These results weaken the hypothesis that different problems require routing among test-time reasoning operators; at this level of abstraction, repeated branching is consistently dominant. Finally, we show that unpaired evaluation and treating scoring-pipeline failures as model errors can materially change, and even reverse, comparative conclusions, motivating paired scoring as a standard protocol for test-time-compute evaluation.

Authors · Shengxin Zhang, Xiaomin Wu, Xiyang Wu, Jing Xie

Read on arXiv

Latest papers

All papers →

Latest from METAL LAB

Figures: Shengxin Zhang et al., arXiv:2608.23956, CC BY 4.0