Research map / Adversarial robustness and security / Certified and empirical robustness
On the Tension Between Optimality and Adversarial Robustness in Policy Optimization
Haoran Li, Jiayu Lv, Congying Han, Zicheng Zhang, Anqi Li, Yan Liu, Tiande Guo, Nan Jiang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train with ARPO (strongest adversary) instead of SPO | PPO · MuJoCo (Hopper/Walker2d/HalfCheetah/Ant) | natural return: Hopper 3081 -> 2101; Halfcheetah 5048 -> 1412 | Hurt |
| train with ARPO instead of SPO | PPO · MuJoCo | worst-case return under attack: Ant -3107 -> 1648 | Improved |
| use bilevel KL-surrogate adversary (BARPO) without SPO guidance instead of ARPO (ablation) | ARPO · MuJoCo | natural and worst-case return: natural +75%, +104%, +156%, +113%; worst +4%, +173%, +209%, +61% | Improved |
| add SPO guidance to BARPO (ablation) | BARPO · MuJoCo | natural return vs worst-case robustness | Mixed |
| use BARPO instead of ARPO with SPO guidance (ablation) | ARPO (w g) · MuJoCo | robust return: +19%, +193%, +138% robust return on Hopper, Halfcheetah, Ant | Improved |
| use BAR-PPO instead of WocaR-PPO | PPO · MuJoCo Walker2d | worst-case return: 1064 -> 2699 | Improved |
| use BAR-PPO instead of SA-PPO | PPO · MuJoCo Hopper | worst-case robustness: -0.634 vs -0.636 | No clear effect |
Closest papers by meaning
- Towards Optimal Adversarial Robust Q-learning with Bellman Infinity-error (ICML 2024 · similarity 0.89)
- Robust Adversarial Reinforcement Learning via Bounded Rationality Curricula (ICLR 2024 · similarity 0.88)
- Beyond Worst-case Attacks: Robust RL with Adaptive Defense via Non-dominated Policies (ICLR 2024 · similarity 0.87)
- Adversarial Diffusion for Robust Reinforcement Learning (NeurIPS 2025 · similarity 0.87)
- Robust and Diverse Multi-Agent Learning via Rational Policy Gradient (NeurIPS 2025 · similarity 0.87)
- Off-Policy Actor-Critic for Adversarial Observation Robustness: Virtual Alternative Training via Symmetric Policy Evaluation (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
