Research map / Reinforcement learning / Policy gradient and actor-critic
Balancing Constraints and Rewards with Meta-Gradient D4PG
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Adapt the Lagrange multiplier learning rate with meta-gradients | D4PG with soft constraints · RWRL suite overall (domains, safety coefficients, and thresholds) | penalized return: 432.70 ± 11.99 -> 677.93 ± 25.78 | Helped |
| Adapt the Lagrange multiplier learning rate with meta-gradients | D4PG with soft constraints · RWRL suite overall (domains, safety coefficients, and thresholds) | penalized return: 478.60 ± 89.26 -> 677.93 ± 25.78 | Helped |
| Adapt the Lagrange multiplier learning rate with meta-gradients | D4PG with soft constraints · RWRL suite overall (domains, safety coefficients, and thresholds) | penalized return: 641.41 ± 26.67 -> 677.93 ± 25.78 | Helped |
| Use critic loss alone rather than actor loss as the MetaL outer loss (ablation) | MetaL · RWRL suite experiments | overall performance | Helped |
| Add actor loss to the critic-only MetaL outer loss (ablation) | MetaL · RWRL suite experiments | overall performance | No clear effect |
| Replace critic-only outer loss with actor-only outer loss (ablation) | MetaL · RWRL suite experiments | overall performance | Hurt |
| Use MeSh reward shaping with learned scale and offset instead of MetaL (ablation) | D4PG with soft constraints · RWRL suite experiments | overall performance | Hurt |
Closest papers by meaning
- A Simple Reward-free Approach to Constrained Reinforcement Learning (ICML 2022 · similarity 0.89)
- Constrained Meta Reinforcement Learning with Provable Test-Time Safety (ICML 2026 · similarity 0.89)
- Reinforcement Learning with Convex Constraints (NeurIPS 2019 · similarity 0.89)
- Constrained Reinforcement Learning Under Model Mismatch (ICML 2024 · similarity 0.88)
- Reward Penalties on Augmented States for Solving Richly Constrained RL Effectively (AAAI 2024 · similarity 0.87)
- MAMBA: an Effective World Model Approach for Meta-Reinforcement Learning (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.