AI Paper Atlas

Research map / Reinforcement learning / RL from feedback

Demystifying Entropy Control in LLM RL Training: Theoretical Analysis and Dynamic Scheduling

Jingchu Gai, Guanning Zeng, Huaqing Zhang, Han Zhong, Yige Hong, Andrej Risteski, Aditi Raghunathan

ICML 2026 · Reinforcement learning › RL from feedback · approach: RL for LLM reasoning

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add dynamic entropy scheduling scaled by Entropy Discrepancy to GRPOGRPO · Knights-and-Knaves KNK5, Pass@1Pass@1 gain (percentage points): +17.52 percentage pointsImproved
add dynamic entropy scheduling scaled by Entropy Discrepancy to GRPOGRPO · Math reasoning benchmarks (AIME24 etc.)Pass@K gain (percentage points): up to +6.7 percentage pointsImproved
add positive entropy regularization (coef 0.05) when Entropy Discrepancy is high (ablation)GRPO on Countdown · Countdown, 39 tasksSpearman correlation of gain vs discrepancy: Spearman 0.8039Improved
add negative entropy regularization (coef -0.01) when Entropy Discrepancy is low (ablation)GRPO on Countdown · Countdown, 22 tasksSpearman correlation of gain vs discrepancy: Spearman -0.8729Mixed
lower positive entropy coefficient to 0.01 (ablation)GRPO on Countdown · Countdown, 28 tasksSpearman correlation of gain vs discrepancy: Spearman 0.8749No clear effect
vary sampling temperature among 0.4, 0.7, 1.0 (ablation)Dynamic entropy scheduling · Knights-and-Knavesaccuracy vs vanilla GRPOImproved
apply an identical KL constraint at two coefficients to both methods (ablation)Dynamic entropy scheduling · Knights-and-Knavesaccuracy gain over GRPO: around 10 percentage points on averageImproved
use dynamic schedule instead of grid-searched fixed entropy coefficient (ablation)Entropy regularization · Knights-and-Knaves and math reasoningaccuracyNo clear effect
use dynamic entropy schedule instead of PKPO, Reward Unlikely, CISPORL reasoning methods · math reasoningPass@K / accuracyNo clear effect
static entropy increase vs decrease depends on task (ablation)GRPO · Knights-and-Knaves vs math reasoningPass@1 / Pass@KMixed

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy