Research map / Large language models / Reasoning and chain of thought

EAGer: Entropy-Aware GEneRation for Adaptive Inference-Time Scaling

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Test-time compute scaling

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes EAGer, a training-free decoding method that branches parallel reasoning traces only at high-entropy tokens and reallocates saved sampling budget to hard prompts, improving Pass@k while using fewer tokens.

Problem

Parallel sampling for inference-time scaling allocates the same compute budget to every prompt and regenerates redundant identical prefixes, wasting tokens on easy problems while under-exploring hard ones.

Approach

Use top-K token entropy during generation as an online uncertainty signal: branch a new reasoning trace only at high-entropy tokens (reusing prefixes), stop early on easy prompts, and reallocate the saved budget to saturating or failing prompts, with or without target labels.

Key results

• With target labels (EAGer), Pass@k improves by up to 37% and tokens drop by up to 59% versus Full Parallel on AIME 2025 for SmolLM 3B (0.53 to 0.73 Pass@k).
• Label-free EAGer-adapt on GPT-Oss 20B reaches 0.95 Pass@k on AIME 2025 vs 0.90 for Full Parallel, with 5 vs 10 tokens (x1e5).
• Entropy-threshold calibration on only 10 MATH-500 examples transfers across benchmarks; 27 of 30 validation runs recovered the same threshold on Qwen3-4B.

Surprising findings

• Pass Rate of EAGer-init is equal or better than Full Parallel even though it generates far fewer sequences, while Pass@k drops for some models.
• SmolLM 3B under Full Parallel sampling produces degenerate repetitions (e.g., 'The answer is: The answer is:'), giving near-zero performance.

Limitations

• Full EAGer requires target labels to identify failing prompts, limiting direct use to verifiable tasks.
• Entropy threshold theta is a per-model hyperparameter requiring a small calibration set.
• Evaluated only on models from 3B to 20B and on verifiable tasks; generalization to larger models and open-ended generation untested.

Experimental setup

Datasets: MATH-500 GSM8K ZebraLogic

Benchmarks: AIME 2024 AIME 2025 HMMT 2025 GPQA-Diamond HumanEval Plus

Metrics: Pass@k Cons@k Pass Rate #Tokens #Sequences wall-clock time

Baselines: Full Parallel sampling ESC DeepConf

Builds on: chain-of-thought prompting nucleus sampling RLVR DeepConf ESC

Closest papers by meaning

• Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach (NeurIPS 2025 · similarity 0.89)
• Catch Your Breath: Adaptive Computation for Self-Paced Sequence Production (NeurIPS 2026 · similarity 0.88)
• DiffAdapt: Difficulty-Adaptive Reasoning for Token-Efficient LLM Inference (ICLR 2026 · similarity 0.88)
• ARES: Multimodal Adaptive Reasoning via Difficulty-Aware Token-Level Entropy Shaping (ICLR 2026 · similarity 0.88)
• What If We Allocate Test-Time Compute Adaptively? (ICML 2026 · similarity 0.88)
• Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for LLM Problem-Solving (ICLR 2025 · similarity 0.88)