Research map / Reinforcement learning / Policy gradient and actor-critic
Robust Reinforcement Learning for Continuous Control with Model Misspecification
Daniel J. Mankowitz, Nir Levine, Rae Jeong, Abbas Abdolmaleki, Jost Tobias Springenberg, Yuanyuan Shi, Jackie Kay, Todd Hester, Timothy Mann, Martin Riedmiller
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add robust worst-case transition evaluation | E-MPO · Nine Mujoco domains with unseen perturbations | average reward | Improved |
| Add soft-robust average-transition evaluation | E-MPO · Mujoco domains with unseen perturbations | average reward | Mixed |
| Add robust worst-case transition evaluation | E-MPO · Shadow hand | average reward | Improved |
| Use robust or soft-robust transition evaluation | E-SVG · Cartpole and Pendulum | average reward | Improved |
| Train the non-robust agent for three times more episodes (ablation) | Non-robust agent · Cartpole Balance and Pendulum Swingup | average reward | No clear effect |
| Train with domain randomization over the limited uncertainty-set perturbations (ablation) | Non-robust agent · Cartpole Balance and Pendulum Swingup | average reward | Hurt |
| Increase domain randomization to a larger perturbation set (ablation) | Non-robust agent · Pendulum Swingup | average reward | Mixed |
| Modify the third uncertainty-set perturbation so it approaches the unseen evaluation dynamics (ablation) | R-MPO · Pendulum Swingup | average reward | Improved |
| Add entropy regularization to robust MPO (ablation) | R-MPO · Mujoco domains | average reward | Mixed |
| Use offline-data-trained transition models as the uncertainty set (ablation) | R-MPO · Cartpole Swingup and Pendulum Swingup | average reward | Mixed |
| Increase offline data used to train uncertainty-set transition models (ablation) | DDR-MPO · Cartpole Swingup and Pendulum Swingup | average reward | Mixed |
| Change the nominal model from the smallest to the median or largest perturbation (ablation) | Robust and non-robust agents · Mujoco domains with held-out perturbations | average reward | Mixed |
Closest papers by meaning
- Time-Constrained Robust MDPs (NeurIPS 2024 · similarity 0.89)
- Robust Offline Reinforcement Learning with Linearly Structured $f$-Divergence Regularization (ICML 2025 · similarity 0.89)
- State Entropy Regularization for Robust Reinforcement Learning (NeurIPS 2025 · similarity 0.88)
- Robust Gymnasium: A Unified Modular Benchmark for Robust Reinforcement Learning (ICLR 2025 · similarity 0.88)
- Robust Model Based Reinforcement Learning Using $\mathcal{L}_1$ Adaptive Control (ICLR 2024 · similarity 0.88)
- Model-Free Robust $\phi$-Divergence Reinforcement Learning Using Both Offline and Online Data (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
