Research map / Reinforcement learning / Policy gradient and actor-critic
Differentiable Trust Region Layers for Deep Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use differentiable per-state trust-region projection instead of PPO-M | Gaussian policy optimization · Halfcheetah-v2 | final mean return: 4255\pm17; 3692\pm15; 3552\pm14 vs 3272\pm18 | Helped |
| use differentiable per-state trust-region projection instead of PPO-M | Gaussian policy optimization · Walker2d-v2 | final mean return: 3583\pm14; 3443\pm19; 3390\pm17 vs 2771\pm40 | Helped |
| use differentiable per-state trust-region projection instead of PPO | Gaussian policy optimization · Hopper-v2 | final mean return: 2578\pm17; 2490\pm19; 2476\pm10; PPO 2515\pm15 | No clear effect |
| add entropy control (ablation) | FROB, W2, and KL · Halfcheetah-v2 | final mean return: FROB 3552\pm14 -> 4568\pm14; W2 3692\pm15 -> 4213\pm13; KL 4255\pm17 -> 4584\pm18 | Helped |
| add entropy control (ablation) | FROB, W2, and KL · Ant-v2 | final mean return: FROB 3035\pm26 -> 3475\pm26; W2 3086\pm28 -> 3361\pm30; KL 3335\pm21 -> 3437\pm21 | Helped |
| add entropy control (ablation) | FROB and W2 · Humanoid-v2 | final mean return: FROB 5202\pm23 -> 5807\pm18; W2 5057\pm25 -> 5844\pm8 | Helped |
| add entropy control (ablation) | KL · Humanoid-v2 | final mean return: 5510\pm27 -> 5430\pm16 | Hurt |
| add entropy control (ablation) | FROB, W2, and KL · Hopper-v2 and Walker2d-v2 | final mean return: Hopper: FROB 2578\pm17 -> 2478\pm11; W2 2490\pm19 -> 2437\pm13; KL 2476\pm10 -> 2497\pm16. Walker2d: FROB 3443\pm19 -> 3 | No clear effect |
| use contextual covariance with W2 or KL projection (ablation) | trust-region policy · semi-sparse 5-link Reacher | mean cumulative reward | Helped |
| use contextual covariance with FROB projection (ablation) | trust-region policy · semi-sparse 5-link Reacher | mean cumulative reward | No clear effect |
| use contextual covariance with PPO or PAPI (ablation) | baseline policy optimization · semi-sparse 5-link Reacher | mean cumulative reward | Hurt |
| replace the concurrent regression penalty with separate post-epoch full regression (ablation) | trust-region layer policy update · trust-region constraint satisfaction | Mahalanobis distance to mean bound | No clear effect |
Closest papers by meaning
- Trust Region Masking for Long-Horizon LLM Reinforcement Learning (ICML 2026 · similarity 0.89)
- Embedding Safety into RL: A New Take on Trust Region Methods (ICML 2025 · similarity 0.87)
- Trust Region Reward Optimization and Proximal Inverse Reward Optimization Algorithm (NeurIPS 2025 · similarity 0.87)
- Simple Policy Optimization (ICML 2025 · similarity 0.87)
- Signal-Adaptive Trust Regions for Gradient-Free Optimization of Recurrent Spiking Neural Networks (NeurIPS 2026 · similarity 0.86)
- An Implicit Trust Region Approach to Behavior Regularized Offline Reinforcement Learning (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.