Research map / Reinforcement learning / Policy gradient and actor-critic
Model-based Safe Deep Reinforcement Learning via a Constrained Proximal Policy Optimization Algorithm
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MBPPO-Lagrangian instead of model-free PPO-Lagrangian | Constrained RL on PointGoal and CarGoal · PointGoal and CarGoal | environment interactions to reach comparable reward performance: approximately 3-4 times more sample efficient | Helped |
| Use MBPPO-Lagrangian instead of PPO-Lagrangian | Constrained RL on PointGoal and CarGoal · PointGoal and CarGoal | cumulative hazard violations: reduces its cumulative hazard violations by ~60% | Helped |
| Use MBPPO-Lagrangian instead of PPO-Lagrangian | Constrained RL · PointGoal and CarGoal | reward return at convergence | No clear effect |
| Use MBPPO-Lagrangian instead of safe-LOOP | Model-based constrained RL · PointGoal and CarGoal | reward performance | Helped |
| Use MBPPO-Lagrangian instead of safe-LOOP | Model-based constrained RL · RC-Car | reward and cost performance | No clear effect |
| Use MBPPO-Lagrangian instead of PPO | Safe RL on DoggoGoal · DoggoGoal | episodic reward: 21.3 ± 1.23 -> -0.69 ± 0.06 | Hurt |
| Increase the safety-threshold multiplier β (ablation) | MBPPO-Lagrangian · PointGoal, CarGoal, and RC-Car | cost returns | Mixed |
| Decrease the safety-threshold multiplier β (ablation) | MBPPO-Lagrangian · PointGoal, CarGoal, and RC-Car | reward returns | Hurt |
| Scale the safety threshold proportionally to the truncated horizon, d := d*H/T (ablation) | Model-based constrained RL · Constrained RL with truncated rollouts | constraint satisfaction | Hurt |
| Use constrained rather than unconstrained exploration for learning the environment model (ablation) | Model-based safe RL · PointGoal1 model learning | validation loss convergence | Hurt |
Closest papers by meaning
- Enforcing Hard Constraints with Soft Barriers: Safe Reinforcement Learning in Unknown Stochastic Environments (ICML 2023 · similarity 0.92)
- Conservative and Adaptive Penalty for Model-Based Safe Reinforcement Learning (AAAI 2022 · similarity 0.92)
- Sampling-Based Safe Reinforcement Learning (NeurIPS 2026 · similarity 0.91)
- Safe Reinforcement Learning with Instantaneous Constraints: The Role of Aggressive Exploration (AAAI 2024 · similarity 0.91)
- Safe Reinforcement Learning using Finite-Horizon Gradient-based Estimation (ICML 2024 · similarity 0.90)
- Constrained Reinforcement Learning Under Model Mismatch (ICML 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.