Research map / Large language models / Reasoning and chain of thought

Inference Time Optimization with Confidence Dynamics

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Test-time compute scaling · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Confidence Dynamic Gain (CDG) voting, a training-free best-of-N answer selection method that rewards traces whose token confidence rises from head to tail, and reports gains over majority voting and DeepConf on AIME/HMMT/BRUMO.

Problem

Majority voting and static confidence-weighted voting for best-of-N answer selection ignore how the model's confidence evolves along a reasoning trace, leaving a discriminative signal unused.

Approach

Score each trace by mean token confidence plus beta times the confidence dynamic gain (tail-minus-head position-normalized confidence), and aggregate answers with count^alpha times mean trace score; a GRPO-based theory is offered to explain why correct traces show positive gain.

Key results

• CDG achieves 74.6% overall average accuracy vs 69.2% for majority voting and 72.9% for DeepConf-Tail across 16 model-benchmark configurations (Table 2).
• On DeepSeek-R1-8B, CDG reaches 90.8% average vs 84.2% for majority voting.
• Removing the start-confidence subtraction ('No Start') drops overall accuracy from 74.6% to 70.2%.

Surprising findings

• Excluding \boxed{} tokens from the confidence trajectory leaves CDG answer selection essentially unchanged (99% per-question agreement).
• For Gemma-3-27B on AIME24, wrong traces do not show a significant frequency of negative CDG values; the separation is attenuated gains rather than declines.

Limitations

• beta is model-dependent and requires a small per-model validation set to estimate.
• Requires top-K token logprobs at every decoding step, unavailable from some closed-source APIs.
• Evaluation limited to math benchmarks plus a partial GPQA-Diamond study; the theory relies on simplifying assumptions (M1-M3) and a toy model.

Experimental setup

Datasets: AIME 2024 AIME 2025 HMMT 2025 BRUMO 2025 GPQA-Diamond

Benchmarks: AIME 2024 AIME 2025 HMMT 2025 BRUMO 2025 GPQA-Diamond

Metrics: accuracy Wilcoxon signed-rank p-value Cohen's d

Baselines: Majority Voting DeepConf-Mean DeepConf-Tail Pass@1

Builds on: self-consistency DeepConf GRPO Dynamic Voting

Closest papers by meaning

• Sample Smart, Not Hard: Correctness-First Decoding for Better Reasoning in LLMs (ICLR 2026 · similarity 0.90)
• Diagnosing Multi-step Reasoning Failures in Black-box LLMs via Stepwise Confidence Attribution (ICML 2026 · similarity 0.89)
• Consilience for Verifier-Free Test-Time Scaling (NeurIPS 2026 · similarity 0.89)
• A Theoretical Study on Bridging Internal Probability and Self-Consistency for LLM Reasoning (NeurIPS 2025 · similarity 0.89)
• From Drift to Coherence: Stabilizing Beliefs in LLMs (ICML 2026 · similarity 0.89)
• Fixing the Broken Compass: Diagnosing and Improving Inference-Time Reward Modeling (ICLR 2026 · similarity 0.89)