Research map / Reinforcement learning / Policy gradient and actor-critic
Responsive Safety in Reinforcement Learning by PID Lagrangian Methods
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add proportional control to integral Lagrangian control (PI control) (ablation) | Constraint-controlled PPO · Safety Gym, including DoggoGoal2 and DoggoButton1 | cost figure of merit | Helped |
| Add proportional control to integral Lagrangian control (ablation) | Constraint-controlled PPO · DoggoButton1 | episodic cost oscillations | Helped |
| Increase the integral-control learning rate to K_I=10^{-1} instead of using moderate K_I=10^{-3} (ablation) | Integral-only constraint-controlled PPO · DoggoButton1 | returns | Hurt |
| Add derivative cost control (ablation) | Integral- and PI-controlled PPO · DoggoButton1 | cost overshoot | Helped |
| Add derivative cost control (ablation) | Constraint-controlled PPO · DoggoButton1 | rate of cost increase within feasible regions | Helped |
| Use gradient-based objective scaling instead of leaving the cost objective unscaled (β=1) (ablation) | PI-controlled PPO · PointGoal1 with reward scales 0.1, 1, and 10 | learning-dynamics consistency across reward scales | Helped |
| Use integral-only control instead of adding proportional control (ablation) | Constraint-controlled PPO · Safety Gym | cost oscillations and overshoot | Hurt |
| Use fast integral control instead of PI control (ablation) | Constraint-controlled PPO · DoggoButton1 | control stability | Hurt |
Closest papers by meaning
- Safe Reinforcement Learning with Instantaneous Constraints: The Role of Aggressive Exploration (AAAI 2024 · similarity 0.89)
- Safe Reinforcement Learning using Finite-Horizon Gradient-based Estimation (ICML 2024 · similarity 0.89)
- Balance Reward and Safety Optimization for Safe Reinforcement Learning: A Perspective of Gradient Manipulation (AAAI 2024 · similarity 0.88)
- Long-Term Safe Reinforcement Learning with Binary Feedback (AAAI 2024 · similarity 0.88)
- Reinforcement Learning with Adaptive Regularization for Safe Control of Critical Systems (NeurIPS 2024 · similarity 0.88)
- Sampling-Based Safe Reinforcement Learning (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.