Research map / Large language models / Reasoning and chain of thought

Confidence-Calibrated Inference Expansion for Evaluator-Guided Test-Time Reasoning

NeurIPS 2026 · Large language models › Reasoning and chain of thought · approach: Efficient and adaptive thinking

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a compute-aware three-action controller (trust, fallback, expand) for evaluator-guided test-time reasoning, using calibrated per-action confidence heads and an uncertainty penalty to cut compute on width expansion and revision tasks.

Problem

Evaluator-guided test-time reasoning needs a local decision at each step on whether to trust the current answer, fall back to a conservative answer, or buy another inference step, under explicit compute cost.

Approach

Scores each action by an action-specific calibrated success probability (terminal actions: answer success; expand: rollout-based continuation-success surrogate) minus a priced compute cost, with a local uncertainty penalty, and gives local action-gap and trajectory-regret analysis for when score errors change decisions.

Key results

• Width expansion: utility 0.715 at normalized compute 0.547 vs 0.707 at 0.781 for calibration-selected fixed policy (about 30% compute reduction).
• Revision: utility 0.673 at compute 0.529 vs 0.668 at 0.632 for best fixed-depth revision (about 16% compute reduction).
• Four-seed paired comparison with fitted Q: compute lower in all 8 seed-scenario comparisons, while utility differences are small and mixed (e.g., width -0.001 CI [-0.006, 0.004]).

Surprising findings

• Utility differences from fitted Q are small and not consistently positive; the gains are mainly in compute rather than accuracy.
• When compute is free, width utility is 0.003 below the best fixed reference.

Limitations

• Evaluation limited to three backbones, MATH/AIME25 width expansion, and a curated deterministic-answer revision benchmark.
• Open-ended generation and long-horizon interaction untested.
• Controller requires refitting and recalibration under changes to model, evaluator, or prompts.

Experimental setup

Datasets: MATH AIME25 OpenThoughts-114k

Benchmarks: MATH AIME25 General Reasoning (OpenThoughts-derived)

Metrics: accuracy compute-adjusted utility normalized compute ECE

Baselines: Majority vote N=48 Verifier trust N=48 Margin heuristic Uncalibrated confidence Calibrated confidence Fitted-Q controller Rollback heuristic Budget-conditioned value model

Builds on: best-of-N with verifier reranking self-refine critique-and-revision fitted Q iteration post-hoc calibration

Closest papers by meaning

• What If We Allocate Test-Time Compute Adaptively? (ICML 2026 · similarity 0.89)
• LYNX: Learning Dynamic Exits for Confidence-Controlled Reasoning (NeurIPS 2026 · similarity 0.87)
• Beyond Test-Time Memory: State-Space Optimal Control for LLM Reasoning (ICML 2026 · similarity 0.87)
• Conformal Thinking: Risk Control for Reasoning on a Compute Budget (ICML 2026 · similarity 0.86)
• Reasoning Models Better Express Their Confidence (NeurIPS 2025 · similarity 0.86)
• Consilience for Verifier-Free Test-Time Scaling (NeurIPS 2026 · similarity 0.85)