Research map / Reinforcement learning / Policy gradient and actor-critic
Leverage the Average: an Analysis of KL Regularization in Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add KL regularization (MD/DA-VI variants) to DQN | DQN · Cartpole and Asterix | average undiscounted return | Helped |
| use medium to large KL coefficient instead of small (ablation) | KL-regularized DQN · Cartpole and Asterix | average return | Mixed |
| add regularization to the evaluation step (ablation) | KL-regularized DQN · Cartpole and Asterix | average return | Helped |
| add KL term to the evaluation step when entropy is zero (ablation) | KL-regularized DQN · Cartpole and Asterix | average return | No clear effect |
| use MD-direct instead of MD-indirect greedy step (ablation) | MD-VI · Cartpole and Asterix | average return | Mixed |
| add entropy on top of KL regularization (ablation) | KL-regularized DQN · Cartpole and Asterix | average return | No clear effect |
| use KL regularization (lambda>0) instead of plain AVI (ablation) | approximate value iteration · simple tabular experiment | relative q error | Helped |
Closest papers by meaning
- Sharp Analysis for KL-Regularized Contextual Bandits and RLHF (NeurIPS 2025 · similarity 0.93)
- Logarithmic Regret for Online KL-Regularized Reinforcement Learning (ICML 2025 · similarity 0.91)
- Fast Rates for Offline Contextual Bandits with Forward-KL Regularization under Single-Policy Concentrability (NeurIPS 2026 · similarity 0.90)
- Demonstration-Regularized RL (ICLR 2024 · similarity 0.89)
- State Entropy Regularization for Robust Reinforcement Learning (NeurIPS 2025 · similarity 0.89)
- Iteratively Refined Behavior Regularization for Offline Reinforcement Learning (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.