Research map / Large language models / Reasoning and chain of thought

Test-Time Scaling in the Wild: Why Exploitation, Not Exploration, Is the Bottleneck

NeurIPS 2026 · Large language models › Reasoning and chain of thought · approach: Test-time compute scaling · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Compute-normalised comparison of five test-time scaling families (Best-of-N, beam search, particle filtering, sequential refinement, fusion) on five open-ended generation benchmarks, finding that exploitation rather than exploration is the bottleneck.

Problem

Test-time scaling has been developed on verifiable tasks (math, code, multiple choice); it is unclear which TTS methods work on open-ended generation where reward models are weak and supervision is scarce.

Approach

Frames TTS methods by how they split a fixed token budget between exploration (candidate generation) and exploitation (selection or synthesis), introduces a bias-corrected oracle estimator, and shows that headroom capture for Best-of-N approximately equals verifier correlation with ground truth.

Key results

• Fusion is the only method improving over the single-sample baseline on every benchmark, reaching 0.610 overall at XHigh vs 0.574 baseline and 0.584 for best RM-based BoN (Qwen3.5-35B-A3B).
• Verifier-RM Spearman correlation with judge scores averages about 0.12 (Skywork) and 0.11 (Llama) on open-ended tasks, and headroom capture for BoN tracks this (slope 1.198, R^2 0.66).
• Sequential Refinement improves on only one of five benchmarks (PRBench, 0.342 vs 0.331 for Fusion), regressing on HealthBench (-2.3pp) and LEXam (-4.5pp).

Surprising findings

• Both tested outcome reward models yield nearly identical realised scores, indicating a structural rather than model-specific verifier failure.
• OLMo3.1-32B-Think's fusion output is worse than a random candidate on two of three benchmarks, showing generation capability does not imply synthesis/exploitation capability.

Limitations

• Conclusions rest on two model families and a single unified judge (Qwen3.5-397B-A17B), with judge-agreement validation only on samples.
• Oracle estimator assumes i.i.d. candidates, which is only approximate for refinement, fusion and particle filtering.
• WritingBench results are confounded by verbosity bias in the evaluation design.

Experimental setup

Datasets: LEXam HealthBench PRBench WildBench WritingBench

Benchmarks: LEXam HealthBench PRBench WildBench WritingBench

Metrics: realised quality oracle quality headroom capture Spearman correlation between RM and judge scores Macro F1 judge agreement Cohen's kappa QWK

Baselines: Best-of-N with Skywork-Reward-V2 Best-of-N with Llama-3.1-70B-Instruct-RM-RB2 Beam Search Particle Filter Sequential Refinement Budget Forcing

Builds on: Best-of-N sampling particle filtering for LLM reasoning budget forcing sequential self-refinement

Closest papers by meaning

• Provable Scaling Laws for the Test-Time Compute of Large Language Models (NeurIPS 2025 · similarity 0.91)
• Sample, Scrutinize and Scale: Effective Inference-Time Search by Scaling Verification (ICML 2025 · similarity 0.91)
• ATTS: Asynchronous Test-Time Scaling via Conformal Prediction (ICLR 2026 · similarity 0.90)
• Every Rollout Counts: Optimal Resource Allocation for Efficient Test-Time Scaling (NeurIPS 2025 · similarity 0.90)
• Rethinking Fine-Tuning when Scaling Test-Time Compute: Limiting Confidence Improves Mathematical Reasoning (NeurIPS 2025 · similarity 0.90)
• AgentTTS: Large Language Model Agent for Test-time Compute-optimal Scaling Strategy in Complex Tasks (NeurIPS 2025 · similarity 0.90)