Research map / Large language models / Reasoning and chain of thought

Mitigating Premature Exploitation in Particle-based Monte Carlo for Inference-Time Scaling

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Test-time compute scaling · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Entropic Particle Filtering (ePF), which adds entropy-driven temperature annealing and one-step look-ahead modulation to particle filtering guided by a PRM, to mitigate premature exploitation in inference-time scaling for math reasoning.

Problem

Particle filtering guided by overconfident process reward models collapses early (particle impoverishment), prematurely pruning correct reasoning paths and yielding suboptimal solutions under tight compute budgets in math reasoning.

Approach

Entropic Annealing dynamically raises the resampling temperature when ESS is low (flattening weights to preserve diversity), and Look-ahead Modulation reweights particles by the PRM score of a one-step sampled successor to make resampling less myopic.

Key results

• On AIME 2025 with Qwen3-1.7B (no thinking, 12k tokens), ePF reaches 38.9 vs 26.6 for PF and 28.8 for BoN.
• ePF w/ LaM improves over PF on AIME 2025 with Qwen3-0.6B (aggregate 25.16 vs 20.25 for PF, proportional weighting).
• On the Qwen2.5-1.5B MATH500 benchmark, ePF reaches 66.42 Top-1 vs 60.15 for PF.

Surprising findings

• ePF's step-wise reward initially dips below PF's as entropic annealing forces exploration, yet final task reward is higher.
• On AIME 2024 with Qwen2.5-7B, PF beats ePF under the uniform and inverse-weighted aggregates (e.g., 21.60 vs 21.00 uniform).

Limitations

• Advantage is largest at small particle budgets and diminishes as N grows.
• Look-ahead Modulation adds extra forward passes per resampling step.
• Cannot correct a consistently inaccurate reward model; limited to PRM quality.

Experimental setup

Datasets: GSM8K MATH500 DeepMath OmniMath AIME 2024 AIME 2025

Benchmarks: GSM8K MATH500 DeepMath OmniMath AIME 2024 AIME 2025

Metrics: Top-1 accuracy pass@1 effective sample size resampling weight variance

Baselines: Self-Consistency Best-of-N Beam Search Particle Filtering TSMC IAS-C CoT DVTS

Builds on: Particle Filtering for LLM posterior estimation Auxiliary Particle Filter Sequential Importance Resampling Soft Actor-Critic

Closest papers by meaning

• Rollout Roulette: A Probabilistic Inference Approach to Inference-Time Scaling of LLMs using Particle-Based Monte Carlo Methods (NeurIPS 2025 · similarity 0.91)
• Self-Rewarding Sequential Monte Carlo for Masked Diffusion Language Models (NeurIPS 2026 · similarity 0.88)
• Scalable Power Sampling: Unlocking Efficient, Training-Free Reasoning for LLMs via Distribution Sharpening (ICML 2026 · similarity 0.87)
• UnMaskFork: Test-Time Scaling for Masked Diffusion via Deterministic Action Branching (ICML 2026 · similarity 0.86)
• Adaptive Inference-Time Scaling via Cyclic Diffusion Search (NeurIPS 2025 · similarity 0.86)
• Entropy-Gated Latent Recursion (NeurIPS 2026 · similarity 0.86)