Research map / Large language models / Reasoning and chain of thought

Entropy-Gated Latent Recursion

NeurIPS 2026 · Large language models › Reasoning and chain of thought · approach: Test-time compute scaling · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes EGLR, a training-free decoding method that entropy-gates recursive re-application of a frozen model's top-L layers, creating a deterministic layer-span axis that combined with temperature sampling yields an L×T rollout pool for self-consistency.

Problem

Inference-time scaling methods such as self-consistency and best-of-N derive rollout diversity only from stochastic token-level sampling, which limits the diversity of the candidate pool.

Approach

Identify the layer span L at which a frozen model's top decoder layers are recursively re-applied at high-entropy tokens as a second, fully deterministic axis of rollout diversity; combine it with temperature to form an L×T Cartesian rollout space, aggregated by majority vote (EGLR-SC).

Key results

• On MATH-500 with Qwen2.5-3B-Instruct, the joint L×T oracle reaches 91.6%, versus 83.4% for the temperature-only oracle and 81.2% for the layer-only oracle.
• EGLR-SC (majority vote over L) improves over greedy on 44/48 model-dataset cells, e.g. Llama-3.1-8B on MATH-500 from 44.8% to 58.0%.
• EGLR-SC beats FLOP-matched Beam-11 on 11 of 12 comparison cells in the Qwen2.5-3B and 7B table.

Surprising findings

• Varying the layer span alone with greedy decoding produces 10-30 pp higher oracle accuracy than the best single L, showing diversity without stochasticity.
• Fusion weight alpha above 0.5 causes sharp accuracy collapse (e.g., 0.9 yields 0.6% on MATH-500).

Limitations

• Plain majority voting recovers only about 28% of the oracle gap, so aggregation is far from the oracle ceiling.
• The full L×T characterization is limited to one model-dataset pair (MATH-500, Qwen2.5-3B) due to compute.
• KL early exit often saturates at K_max=3, so the stopping criterion is not well-calibrated.

Experimental setup

Datasets: GSM8K MATH-500 MinervaMath AMC23 AIME24 AIME25

Benchmarks: GSM8K MATH-500 MinervaMath AMC23 AIME24 AIME25

Metrics: exact-match accuracy oracle accuracy compute cost relative to greedy

Baselines: Greedy decoding Self-consistency Beam search (Beam-4, Beam-11) Entropy-Gated Branching

Builds on: self-consistency Hierarchical Reasoning Model Tiny Recursive Model Looped Transformers Entropy-Gated Branching

Closest papers by meaning

• Rollout Roulette: A Probabilistic Inference Approach to Inference-Time Scaling of LLMs using Particle-Based Monte Carlo Methods (NeurIPS 2025 · similarity 0.90)
• XTC: Head-Aware Sampling by Excluding Top Choices (NeurIPS 2026 · similarity 0.89)
• OptScale: Probabilistic Optimality for Inference-time Scaling (AAAI 2026 · similarity 0.89)
• Recursive Inference Scaling: A Winning Path to Scalable Inference in Language and Multimodal Systems (NeurIPS 2025 · similarity 0.89)
• Sample Smart, Not Hard: Correctness-First Decoding for Better Reasoning in LLMs (ICLR 2026 · similarity 0.88)
• Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for LLM Problem-Solving (ICLR 2025 · similarity 0.88)