Research map / Reinforcement learning / Visual and representation RL
Robust Gymnasium: A Unified Modular Benchmark for Robust Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Robust-Gymnasium, a modular benchmark with over 60 task environments and disruptors over observations, rewards, actions, and environment dynamics, and benchmarks standard, robust, safe, and multi-agent RL algorithms on it.
Problem
Robust RL methods are typically evaluated on one-off environments and target a single type of disruption; there is no standardized benchmark covering disruptions across all stages of agent-environment interaction.
Approach
A unified modular framework (Disrupted-MDP) with observation, action, and environment disruptors operating in four modes (random, adversarial, internal dynamic shift, external disturbance) at configurable frequencies, applied across many task bases including Gymnasium, Robosuite, Fetch, and MAMuJoCo; also includes an LLM-based adversarial state perturbation.
Key results
- PPO and SAC performance degrades quickly as Gaussian state/action disturbance levels increase, especially in post-training evaluation on HalfCheetah-v4.
- RSC shows greater robustness than ATLA and DBC on Robosuite DoorCausal and CausalLift tasks, though with lower training efficiency.
- LLM-based adversarial state perturbations cause a larger PPO performance drop on Ant-v4 than uniform noise, and higher attack frequency (every 100 steps) degrades performance more.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase random observation-state or action disturbance | PPO and SAC · HalfCheetah-v4 | episode reward | Hurt |
| Apply disturbances only during testing rather than during training | PPO and SAC · HalfCheetah-v4 | episode reward | Hurt |
| Add random reward disturbance during training, then evaluate without attack | PPO · HalfCheetah-v4 | reward performance | Helped |
| Apply a disturbance attack | SAC · HalfCheetah-v4 | performance | Hurt |
| Apply internal dynamic shifts to gravity and wind | OMPO · Ant-v5 | performance | Hurt |
| Apply internal dynamic shifts to robot shape | OMPO · Hopper-v5 | performance | Hurt |
| Use RSC instead of ATLA or DBC under external semantic disturbance | Robosuite DoorCausal and LiftCausal · DoorCausal-v1 and LiftCausal-v1 | reward | Helped |
| Add action or observed-cost disturbance during training | PCRPO · Walker2d | performance | Helped |
| Add action or observed-cost disturbance during training | CRPO · Walker2d | performance | Hurt |
| Apply state, action, or reward disruption to all agents | MAPPO and IPPO · MA-HalfCheetah-v4 | performance | Hurt |
| Use partial rather than full state, action, or reward attacks | MAPPO · MA-HalfCheetah-2x3 | performance | Helped |
| Use LLM-based state attacks instead of uniform noise | PPO · Ant-v4 | performance | Hurt |
Surprising findings
- PCRPO under disturbance sometimes outperforms its own undisturbed performance, suggesting that training with appropriate disturbances may help.
- Policies trained with reward disturbances and tested in clean environments may show improved reward performance at test time.
Limitations
- Robust RL baselines are evaluated on only a few representative tasks rather than the full benchmark.
- The LLM-based adversary relies on an external API call per step, which is not detailed for cost or reproducibility.
Experimental setup
Datasets: Gymnasium-MuJoCo Gymnasium-Box2D Robosuite Fetch Franka Kitchen Dexterous Hand Adroit HumanoidBench Safety MuJoCo MAMuJoCo
Benchmarks: HalfCheetah-v4 Ant-v5 Hopper-v5 Ant-v4 Walker2d DoorCausal-v1 LiftCausal-v1 MA-HalfCheetah-v4
Metrics: episode reward safety cost
Baselines: PPO SAC OMPO RSC ATLA DBC PCRPO CRPO MAPPO IPPO
Builds on: Gymnasium Robosuite OMPO RARL CRPO
Closest papers by meaning
- Time-Constrained Robust MDPs (NeurIPS 2024 · similarity 0.90)
- Empirical Study on Robustness and Resilience in Cooperative Multi-Agent Reinforcement Learning (NeurIPS 2025 · similarity 0.90)
- Distributionally Robust Reinforcement Learning with Interactive Data Collection: Fundamental Hardness and Near-Optimal Algorithms (NeurIPS 2024 · similarity 0.90)
- Game-Theoretic Robust Reinforcement Learning Handles Temporally-Coupled Perturbations (ICLR 2024 · similarity 0.89)
- Adversarial Diffusion for Robust Reinforcement Learning (NeurIPS 2025 · similarity 0.89)
- Towards Robust Offline Reinforcement Learning under Diverse Data Corruption (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.