Research map / Reinforcement learning / Policy gradient and actor-critic
Understanding the Impact of Entropy on Policy Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add entropy regularization and decay it during optimization | Exact-gradient policy optimization · 5×5 Gridworld | proportion of sub-optimal solutions found: 25% -> 0 | Helped |
| Increase policy standard deviation above 0.1 (ablation) | REINFORCE · Hopper | learning speed | Helped |
| Increase policy standard deviation above 0.1 (ablation) | REINFORCE · Walker2d | final mean reward: 2 to 8 times larger | Helped |
| Increase policy standard deviation above 0.1 (ablation) | REINFORCE · HalfCheetah | final reward | No clear effect |
| Increase policy standard deviation from 0.1 to 1 (ablation) | Policy optimization with constant learning rates · Hopper and Walker | best constant learning rate: 10 times larger | Helped |
| Increase policy standard deviation (ablation) | Policy optimization · HalfCheetah | curvature fluctuation | No clear effect |
| Increase policy standard deviation from 0.1 to 1.0 (ablation) | Hopper final-solution optimization · Hopper | directions with negative curvature: roughly 3 times more | Helped |
| Evaluate the same solution with a more stochastic policy (ablation) | Hopper policy objective · Hopper | directions with detectable negative curvature: 84% | Helped |
| Increase policy standard deviation (ablation) | Policy objective · Hopper and Walker2d | existence of an increasing interpolation path to a better solution | Helped |
| Increase policy standard deviation (ablation) | Policy objective · HalfCheetah | objective landscape change | No clear effect |
Closest papers by meaning
- Convergence Theorems for Entropy-Regularized and Distributional Reinforcement Learning (NeurIPS 2025 · similarity 0.90)
- State Entropy Regularization for Robust Reinforcement Learning (NeurIPS 2025 · similarity 0.89)
- Intrinsic Benefits of Categorical Distributional Loss: Uncertainty-aware Regularized Exploration in Reinforcement Learning (NeurIPS 2025 · similarity 0.88)
- When Maximum Entropy Misleads Policy Optimization (ICML 2025 · similarity 0.88)
- Global Optimality for Constrained Exploration via Penalty Regularization (NeurIPS 2026 · similarity 0.87)
- Behavioral Entropy-Guided Dataset Generation for Offline Reinforcement Learning (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.