Research map / Reinforcement learning / Policy gradient and actor-critic
SHAPO: Sharpness-Aware Policy Optimization for Safe Exploration
Kaustubh Mani, Yann Pequignot, Vincent Mai, Liam Paull
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add SHAPO to baseline on-policy algorithms (CPO, PIDLag, CRPO, SauteRL, TRPOLag) | on-policy safe RL baselines · Safety Gym and MuJoCo tasks | safety-efficiency tradeoff | Improved |
| add SHAPO to PIDLag across initial multipliers | PIDLag · PointGoal1 lambda_init sweep | return vs cost Pareto frontier | Improved |
| add SHAPO to suppress heavy cost tails | baselines · Ant-v4, Walker2d-v4 | episodic cost tail percentile | Improved |
| use Fisher instead of Euclidean perturbations (ablation) | SHAPO · Safety Gym and MuJoCo tasks | safety and return | Improved |
| add SAM on critic only (ablation) | baseline · Safety Gym and MuJoCo tasks | cost | Hurt |
| add SHAPO on actor only (ablation) | baseline · Safety Gym and MuJoCo tasks | safety | Improved |
| add SAM on critic to SHAPO actor (ablation) | SHAPO Actor · Safety Gym and MuJoCo tasks | safety and efficiency | Improved |
| use fixed pessimism level instead of fixed delta_Down schedule (ablation) | SHAPO · Safety Gym tasks | return and cost | No clear effect |
| increase delta_Down too large (ablation) | SHAPO · hyperparameter sensitivity | return and cost | Hurt |
| decrease delta_Down too small (ablation) | SHAPO · hyperparameter sensitivity | return and cost | No clear effect |
| vary rho_critic over wide range (ablation) | SHAPO · hyperparameter sensitivity | return and cost | No clear effect |
| add SHAPO perturbation step (compute overhead) | PIDLag · PointGoal1 | total runtime: 85 min -> 100 min | Hurt |
Closest papers by meaning
- Safe Exploration via Policy Priors (ICLR 2026 · similarity 0.90)
- Sampling-Based Safe Reinforcement Learning (NeurIPS 2026 · similarity 0.89)
- Safeguarded Progress in Reinforcement Learning: Safe Bayesian Exploration for Control Policy Synthesis (AAAI 2024 · similarity 0.89)
- Enhancing Efficiency of Safe Reinforcement Learning via Sample Manipulation (NeurIPS 2024 · similarity 0.88)
- Safety Representations for Safer Policy Learning (ICLR 2025 · similarity 0.88)
- ActSafe: Active Exploration with Safety Constraints for Reinforcement Learning (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
