AI Paper Atlas

Research map / Large language models / Reasoning and chain of thought

StepBack: Step-Level Error-Localized Resampling for Efficient Test-Time Reasoning

Yang Ouyang, Jung-Eun Kim

NeurIPS 2026 · Large language models › Reasoning and chain of thought · approach: Test-time compute scaling · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes StepBack, a step-level error-localized resampling method that improves efficiency of test-time chain-of-thought reasoning relative to full-trajectory self-consistency.

Problem

Self-consistency for chain-of-thought reasoning requires sampling many complete trajectories under a preset budget, and existing efficient variants only stop early or reallocate samples across questions.

Approach

Instead of resampling whole trajectories, localize errors to specific reasoning steps and resample from the point of error, reducing cost while keeping the reasoning quality of self-consistency.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
replace full-trajectory SC with step-level error-localized resampling (PRM signal)StepBack, 3 models · 7 reasoning and QA benchmarksaccuracy and tokens: +0.36/+0.17/+0.82 points with 18.66/21.07/22.80% fewer tokensImproved
use NLL signal instead of PRM for localization (ablation)StepBack · 7 benchmarkstoken saving: 27.74/27.15/33.32% saved, accuracy -0.17/+0.12/+0.55Improved
oracle first-error localization for suffix resampling (ablation)draft-suffix resampling · GSM8Kaccuracy: 91.21 -> 91.89 with 64.0% savingsImproved
oracle first-error localization for suffix resampling (ablation)draft-suffix resampling · MATH500accuracy: 72.80 -> 72.40 (8x4), 73.00 (16x2)No clear effect
localization on harder benchmarks AIME24StepBack · AIME24accuracy: -2.22 for Llama-3B and Qwen-7BHurt
combine StepBack with Adaptive ConsistencyAC, Qwen2.5-3B · GSM8Kaccuracy: 91.13 -> 92.34; tokens 3,509 -> 2,166Improved
combine StepBack with AC on MATH500AC, Qwen2.5-3B · MATH500accuracy: 72.80 -> 72.60; tokens 11,386 -> 8,749No clear effect
use 1.5B PRM instead of 7B PRM (ablation)StepBack · MATH500accuracy vs SC: +0.36 at 8x4; -0.36 at 16x2 (7B: +0.72)Mixed
more suffix-heavy budget split (1x32, 2x16) (ablation)StepBack · 7 benchmarksaccuracy stability and tokensMixed
more draft-heavy budget split (8x4, 16x2) (ablation)StepBack · 7 benchmarksaccuracyImproved
PRM localization vs final-step resampling (ablation)StepBack · HotpotQAaccuracy and tokensHurt
PRM localization vs final-step resampling at 4x8 (ablation)StepBack · 7 benchmarksaccuracy: 2.06 points more accurateImproved

Experimental setup

Baselines: self-consistency

Builds on: self-consistency

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy