Research map / Reinforcement learning / Policy gradient and actor-critic
On Entropy Control in LLM-RL Algorithms
Han Shen
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use AEnt instead of GRPO | Qwen2.5-math-1.5b (MATH train) · AIME24 | test score: 0.192 -> 0.217 | Improved |
| use AEnt instead of GRPO | DeepSeek-R1-distilled-Qwen-1.5b (OpenR1-math train) · AMC | test score: 0.769 -> 0.825 | Improved |
| use AEnt instead of GRPO | Qwen2.5-math-1.5b (MATH train) · MATH-500 | test score: 0.756 -> 0.750 | Hurt |
| use AEnt instead of conventional entropy regularization (EntReg) | DeepSeek-R1-distilled-Qwen-1.5b (OpenR1-math train) · MATH-500 | test score: 0.872 -> 0.882 | Improved |
| use EntReg instead of GRPO | Qwen2.5-math-1.5b (MATH train) · AIME24 | test score: 0.192 -> 0.167 | Hurt |
| use EntReg instead of GRPO | DeepSeek-R1-distilled-Qwen-1.5b (OpenR1-math train) · MATH-Hard | test score: 0.773 -> 0.808 | Improved |
| add adaptive entropy coefficient instead of constant coefficient (ablation) | AEnt · test score and response length | test score: similar test score; more compact responses | No clear effect |
| add adaptive entropy coefficient instead of constant coefficient (ablation) | AEnt · policy entropy stability | entropy / response length | Improved |
| increase entropy clamping percentage p (ablation) | AEnt · test score | test score: performance drops beyond a point | Mixed |
| vary clamping percentage p (ablation) | AEnt · test score vs baselines | test score | No clear effect |
| use AEnt instead of GRPO | Qwen2.5-math-1.5b (MATH train) · training GPU hours to reach GRPO score | GPU hours: 57 h -> 35 h | Improved |
| use AEnt instead of GRPO | Qwen2.5-math-1.5b (MATH train) · training GPU hours to highest own score | GPU hours: 57 h -> 69 h | Hurt |
Closest papers by meaning
- The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoning (NeurIPS 2025 · similarity 0.89)
- AEM: Adaptive Entropy Modulation for Multi-Turn Agentic Reinforcement Learning (NeurIPS 2026 · similarity 0.88)
- SCOPE-RL: Stable and Quantitative Control of Policy Entropy in RL Post-Training (NeurIPS 2026 · similarity 0.87)
- SFT-then-RL Outperforms Mixed-Policy Methods for LLM Reasoning (NeurIPS 2026 · similarity 0.87)
- State Entropy Regularization for Robust Reinforcement Learning (NeurIPS 2025 · similarity 0.87)
- Convergence Theorems for Entropy-Regularized and Distributional Reinforcement Learning (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
