Research map / Reinforcement learning / MDPs and planning
Saute RL: Almost Surely Safe Reinforcement Learning Using State Augmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| augment the state with remaining safety budget and reshape costs | TRPO on pendulum swing-up · Pendulum swing-up | safety constraint satisfaction | Helped |
| augment the state with remaining safety budget and reshape costs | TRPO on double pendulum · Double pendulum | task cost | Helped |
| use average-constrained optimization instead of almost-sure safety | TRPO-L and CPO · Double pendulum | safety constraint satisfaction | Hurt |
| wrap model-based RL with state augmentation and cost shaping | MBPO and PETS · Pendulum swing-up | task performance | Mixed |
| train by sampling the initial safety budget from [5, 100] | Saut8 TRPO · Double pendulum, budgets 40 and 80 | safety constraint satisfaction | Helped |
| add state augmentation to the full method (ablation) | Saut8 TRPO · Double pendulum | task cost | Helped |
| add state augmentation to the full method (ablation) | Saut8 TRPO · Double pendulum | safety cost | Mixed |
| add cost shaping to the full method (ablation) | Saut8 TRPO · Double pendulum | task cost | Helped |
| add state augmentation to the full method (ablation) | Saut8 TRPO · Pendulum swing-up | task cost | No clear effect |
| increase reshaped-cost penalty n from 0 to 100 (ablation) | Saut8 TRPO · Double pendulum | safety cost outliers: 0 to 100 | Helped |
| increase reshaped-cost penalty n to 10000 and 100000 (ablation) | Saut8 TRPO · Double pendulum | safety and task cost outliers: 10000 and 100000 | Hurt |
| approximate almost-sure constraints with a CVaR constraint | CVaR-L · Reacher | safety constraint satisfaction | Hurt |
Closest papers by meaning
- Long-Term Safe Reinforcement Learning with Binary Feedback (AAAI 2024 · similarity 0.90)
- Probabilistic Shielding for Safe Reinforcement Learning (AAAI 2025 · similarity 0.90)
- Safe Reinforcement Learning with Instantaneous Constraints: The Role of Aggressive Exploration (AAAI 2024 · similarity 0.90)
- Safeguarded Progress in Reinforcement Learning: Safe Bayesian Exploration for Control Policy Synthesis (AAAI 2024 · similarity 0.90)
- A Provable Approach for End-to-End Safe Reinforcement Learning (NeurIPS 2025 · similarity 0.90)
- Flipping-based Policy for Chance-Constrained Markov Decision Processes (NeurIPS 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.