Research map / Large language models / Reasoning and chain of thought

Scalable Power Sampling: Unlocking Efficient, Training-Free Reasoning for LLMs via Distribution Sharpening

ICML 2026 · Large language models › Reasoning and chain of thought · approach: RL with verifiable rewards for reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a training-free, verifier-free autoregressive approximation to power-distribution sampling that uses Monte Carlo lookahead rollouts and jackknife bias correction, matching MCMC power sampling and one-shot GRPO on math/code/QA at over 10x lower latency.

Problem

Sampling from the global power distribution p^alpha of an LLM (which recovers RL-post-training-level reasoning) requires expensive MCMC; this paper seeks a faster sampler that retains its accuracy gains.

Approach

Decomposes the power distribution at each token into a scaled low-temperature distribution times a future-trajectory scaling factor zeta_t, estimated by Monte Carlo lookahead rollouts from the base model and debiased with a jackknife estimator, enabling autoregressive sampling without MCMC.

Key results

• On Qwen2.5-7B, the method reaches 0.708 MATH500, 0.756 HumanEval and 0.349 GPQA pass@1, versus 0.706/0.622/0.318 for MCMC power sampling and 0.740/0.561/0.354 for GRPO (MATH) on MATH500/HumanEval/GPQA.
• On Qwen2.5-Math-7B, the method reaches 0.758 on MATH500, above GRPO (MATH) at 0.785 only on MATH500 gap of 0.027 but better on HumanEval (0.604 vs 0.537) and GPQA (0.409 vs 0.399).
• Inference speedups of up to about 10x over MCMC; on Qwen2.5-Math-7B MATH500 average time falls from 2.5 to 0.22 minutes per prompt.

Surprising findings

• Low-temperature sampling degrades DeepSeek-Math-7B-RL on MATH500 (0.492 to 0.412), suggesting GRPO already sharpens the distribution differently.
• Power sampling underperforms GRPO on MATH500 for Qwen2.5-Math-7B (0.758 vs 0.785), so matching GRPO is not uniform across settings.

Limitations

• Inference cost grows with candidate count K_t and rollout count M_t, adding overhead over standard decoding.
• Sensitive to the power exponent alpha; results depend on alpha choice (fixed at 4).
• Evaluated on only three benchmarks and four 7B models.

Experimental setup

Datasets: MATH500 HumanEval GPQA-diamond

Benchmarks: MATH500 HumanEval GPQA-diamond

Metrics: pass@1 pass@K inference time per prompt

Baselines: Base model Low-temperature sampling Best-of-N MCMC Power Sampling GRPO (MATH)

Builds on: Power Distribution Sampling (MCMC) GRPO low-temperature sampling Best-of-N

Closest papers by meaning

• Power Distribution Bridges Sampling, Self-Reward RL, and Self-Distillation (NeurIPS 2026 · similarity 0.92)
• Reasoning with Sampling: Your Base Model is Smarter Than You Think (ICLR 2026 · similarity 0.91)
• Dynamics-Predictive Sampling for Active RL Finetuning of Large Reasoning Models (ICLR 2026 · similarity 0.91)
• T1: Advancing Language Model Reasoning through Reinforcement Learning and Inference Scaling (ICML 2025 · similarity 0.90)
• Resource-Efficient Reinforcement for Reasoning Large Language Models via Dynamic One-Shot Policy Refinement (ICML 2026 · similarity 0.90)
• Amortizing intractable inference in large language models (ICLR 2024 · similarity 0.90)