Research map / Reinforcement learning / Imitation learning
On Pathologies in KL-Regularized Reinforcement Learning from Expert Demonstrations
Tim G. J. Rudner, Cong Lu, Michael Osborne, Yarin Gal, Yee Whye Teh
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use GP non-parametric behavioral policy instead of parametric NN | KL-regularized RL · MuJoCo tasks | return | Improved |
| use parametric variance functions instead of GP variance (ablation) | N-PPAC · door-binary-v0 | success rate: 20\% vs nearly 100 | Hurt |
| floor the behavioral policy variance (ablation) | parametric behavioral policies · door-binary-v0 | success rate | No clear effect |
| use a single expert demonstration instead of 15 (ablation) | GP behavioral policy · HalfCheetah-v2 | return | No clear effect |
| GP behavioral policy vs parametric, 15,000 data points | KL-regularized RL · HalfCheetah-v2 | time per epoch: 12.00s / 46.54s | Hurt |
| GP behavioral policy vs parametric, 1,000 data points | KL-regularized RL · door-binary-v0 | time per epoch: 19.62s / 23.78s | Hurt |
| alternative divergence metrics instead of KL (ablation) | N-PPAC · MuJoCo and hand tasks | return | Hurt |
| N-PPAC vs AWAC on door opening | N-PPAC · door-binary-v0 | environment interactions to 90\% success: 4$\times$ fewer | Improved |
Closest papers by meaning
- Demonstration-Regularized RL (ICLR 2024 · similarity 0.90)
- Behavior-Regularized Diffusion Policy Optimization for Offline Reinforcement Learning (ICML 2025 · similarity 0.88)
- Iteratively Refined Behavior Regularization for Offline Reinforcement Learning (NeurIPS 2024 · similarity 0.87)
- Uncertainty-Based Smooth Policy Regularisation for Reinforcement Learning with Few Demonstrations (NeurIPS 2025 · similarity 0.86)
- Sharp Analysis for KL-Regularized Contextual Bandits and RLHF (NeurIPS 2025 · similarity 0.86)
- Logarithmic Regret for Online KL-Regularized Reinforcement Learning (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
