Research map / Large language models / Reasoning and chain of thought

Predicting LLM Output Length via Entropy-Guided Representations

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Efficient LLM serving · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes EGTP (entropy-weighted pooling of the LLM's own hidden states with soft-label regression) and PLP (per-step remaining-length prediction), plus a ForeLen benchmark, for LLM output length prediction.

Problem

Output lengths in LLM serving and GRPO-style RL sampling are long-tailed and highly variable, causing padding waste in batched inference; existing auxiliary-model length predictors are costly, generalize poorly, and fail for one-to-many stochastic sampling.

Approach

Reuse the main LLM's own final-layer hidden states, pooled with weights from per-token predictive entropy, and regress length via a soft-label binned head; at decode time, repeat prediction on each step's accumulated states to estimate remaining length.

Key results

• EGTP reduces average MAE on ForeLen to 110.75 (Qwen2.5 3B) vs 146.42 for best baseline TRAIL, about 24% lower; averaged across models the paper reports 29.16% reduction over the best baseline.
• On LMSYS, EGTP MAE is 87.32 for GPT-4 and 68.33 for Claude-2, versus 96.03 and 77.03 for the next-best methods.
• In an end-to-end vLLM SJF setup on Long Sequence, EGTP achieves padding ratio 0.18 vs 0.51 for TRAIL and average JCT 4.20 vs 9.45.

Surprising findings

• Final-layer hidden states perform comparably to intermediate layers for length prediction (Layer 24 RMSE 92.95 vs Layer 12 93.47), which the authors use to justify the final layer.

Limitations

• Ablations and sensitivity to lambda are mostly on LMSYS or a single model (Qwen2.5-7B), limiting generality claims.
• Predictions are based on closed-model results for GPT-4 and Claude-2 with unclear hidden-state access, so the method cannot be applied to them directly in the stated setup.

Experimental setup

Datasets: LongBench ZeroSCROLLS IFEval CRUXEval GSM8K LiveCodeBench MATH MBPP MMLU-STEM LMSYS

Benchmarks: ForeLen LMSYS

Metrics: MAE throughput job completion time padding ratio

Baselines: SSJF-Reg SSJF-MC S3 PiA TPV TRAIL LTR-C

Builds on: BERT SSJF vLLM GRPO

Closest papers by meaning

• Beyond Prediction: Tail-Aware Scheduling for LLM Inference (ICML 2026 · similarity 0.88)
• TokenPowerBench: Benchmarking the Power Consumption of LLM Inference (AAAI 2026 · similarity 0.88)
• Generalized Parallel Scaling with Interdependent Generations (ICLR 2026 · similarity 0.88)
• The Silent Hyperparameter: Quantifying the Impact of Inference Backends on LLM Reproducibility (NeurIPS 2026 · similarity 0.87)
• Provable Length Generalization in Sequence Prediction via Spectral Filtering (ICML 2025 · similarity 0.87)
• Scalable Power Sampling: Unlocking Efficient, Training-Free Reasoning for LLMs via Distribution Sharpening (ICML 2026 · similarity 0.87)