Research map / Large language models / Reasoning and chain of thought

Consilience for Verifier-Free Test-Time Scaling

NeurIPS 2026 · Large language models › Reasoning and chain of thought · approach: Routing, pruning and distillation for reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes consilience, a verifier-free selection score that rewards low initial and high final token confidence in reasoning traces, improving best-of-n selection over confidence-maximization baselines on LCB, HMMT, GPQA and SWE-bench.

Problem

Verifier-free test-time scaling via sequence confidence maximization breaks down on hard problems, where confidently wrong completions are favored; free-form tasks (code, agentic edits) also lack voting.

Approach

Score each completion by S = C_final - alpha*C_initial over boundary windows (after skipping a prefix, and restricted to the reasoning trace), so selection favors traces that explore early (low initial confidence) and converge late (high final confidence).

Key results

• On LiveCodeBench, Cns-think lifts GPT-OSS-120B to 69.7% vs 65.7% Pass@1 and Self-Certainty's 64.5%.
• On HMMT with GPT-OSS-20B, Self-Certainty drops to 68.0% from Pass@1 76.5%, while Cns-2K-think reaches 80.7%.
• On SWE-bench Verified with mini-SWE-agent, consilience raises GPT-OSS-120B from 23.0% to 26.9% resolve rate (n=32).

Surprising findings

• Maximizing mean confidence can degrade accuracy below random selection (Pass@1), e.g., Self-Certainty on HMMT with GPT-OSS-20B drops from 76.5% to 68.0%.
• On hard problems, incorrect completions show higher mean confidence than correct ones, the reverse of the pattern on easy problems.

Limitations

• Consilience is a fixed heuristic; hyperparameters (alpha, window k) were tuned on a development setting and the paper notes it may not be universal.
• Agentic application is costly (18 vs 8 hours) and applied only to edit steps with n=32.
• Gains on easy problems are neutral; improvements on hard problems remain modest in absolute terms (e.g., Hard tier 8.1 vs 3.7 for Qwen).

Experimental setup

Datasets: LiveCodeBench-v6 HMMT25 GPQA-Diamond SWE-bench-Verified AIME25 Natural Questions

Benchmarks: LiveCodeBench-v6 HMMT25 GPQA-Diamond SWE-bench-Verified AIME25 Natural Questions

Metrics: accuracy resolve rate AUROC

Baselines: Pass@1 Self-Certainty DeepConf-20% DeepConf-2K Self-Consistency Self-Refine Self-Verify

Builds on: Self-Certainty DeepConf Self-Consistency

Closest papers by meaning

• Scaling Test-Time Compute Without Verification or RL is Suboptimal (ICML 2025 · similarity 0.91)
• A Theoretical Study on Bridging Internal Probability and Self-Consistency for LLM Reasoning (NeurIPS 2025 · similarity 0.90)
• Rethinking Fine-Tuning when Scaling Test-Time Compute: Limiting Confidence Improves Mathematical Reasoning (NeurIPS 2025 · similarity 0.90)
• Provable Test-Time Scaling for Beam Search in LLM Reasoning (NeurIPS 2026 · similarity 0.89)
• Every Rollout Counts: Optimal Resource Allocation for Efficient Test-Time Scaling (NeurIPS 2025 · similarity 0.89)
• Bag of Tricks for Inference-time Computation of LLM Reasoning (NeurIPS 2025 · similarity 0.89)