Research map / Multi-agent systems and games / Cooperative multi-agent RL
Wolfpack Adversarial Attack for Robust Multi-Agent Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes the Wolfpack adversarial attack, which first hits one agent and then the agents most responsive to it, and WALL, a robust MARL training method that defends against it by training CTDE value-based policies under this attack.
Problem
Robust MARL policies are vulnerable to coordinated adversarial attacks in cooperative settings, since existing attacks typically target a single agent per attack without exploiting the cooperative responses of other agents.
Approach
Attack an initial agent, then identify follow-up agents whose individual Q-distributions change most in response (via KL divergence after a simulated update of Q^tot), and attack them over the next steps; choose attack timing with a Transformer planner that predicts future Q^tot reductions. WALL trains QMIX/VDN/QPLEX policies on this LPA-Dec-POMDP.
Key results
- On MPE, the Wolfpack attack reduces Vanilla QMIX's mean cumulative reward from 455.4 to 367.6, versus 412.8 under EGA and 445.9 under random attack.
- On SMAC under Wolfpack attack, WALL achieves a mean win rate of 93.4% versus 39.4% for Vanilla QMIX and 59.1% for ROMANCE.
- WALL also reaches 99.4% mean win rate in natural (no attack) SMAC settings, slightly above Vanilla QMIX's 98.7%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train with WALL instead of the strongest listed robust baseline | QMIX · MPE Natural, mean across PP scenarios | average cumulative reward: 515.0 \pm 1.4 -> 560.1 \pm 1.7 | Helped |
| Train with WALL instead of the strongest listed robust baseline | QMIX · MPE Random Attack, mean across PP scenarios | average cumulative reward: 499.7 \pm 2.4 -> 553.1 \pm 1.7 | Helped |
| Train with WALL instead of the strongest listed robust baseline | QMIX · MPE EGA, mean across PP scenarios | average cumulative reward: 455.5 \pm 1.5 -> 511.5 \pm 1.4 | Helped |
| Train with WALL instead of the strongest listed robust baseline | QMIX · MPE Wolfpack Attack, mean across PP scenarios | average cumulative reward: 450.9 \pm 8.4 -> 489.5 \pm 0.6 | Helped |
| Train with WALL instead of the strongest listed robust baseline | QMIX · SMAC Natural, mean across scenarios | win rate: 99.3\pm0.2 -> 99.4\pm0.5 | Helped |
| Train with WALL instead of the strongest listed robust baseline | QMIX · SMAC Random Attack, mean across scenarios | win rate: 87.8\pm2.4 -> 95.9\pm0.5 | Helped |
| Train with WALL instead of the strongest listed robust baseline | QMIX · SMAC EGA, mean across scenarios | win rate: 87.5\pm1.6 -> 93.6\pm1.5 | Helped |
| Train with WALL instead of the strongest listed robust baseline | QMIX · SMAC Wolfpack Attack, mean across scenarios | win rate: 63.5\pm3.1 -> 93.4\pm1.1 | Helped |
| Replace Random Attack with the coordinated Wolfpack attack | Vanilla QMIX · MPE, mean across PP scenarios | average cumulative reward: 455.4 \pm 2.1 -> 367.6 \pm 5.0 | Hurt |
| Replace Random Attack with the coordinated Wolfpack attack | Vanilla QMIX · SMAC, mean across scenarios | win rate: 98.7\pm0.5 -> 39.4\pm4.5 | Hurt |
| Select the initial agent by minimizing Qtot instead of random selection (ablation) | Wolfpack attack and WALL · 8m and MMM | robustness | Mixed |
| Select follow-up agents by L2 distance instead of Q-policy change (ablation) | Wolfpack attack and WALL · 8m and MMM | attack severity and robustness | Hurt |
Surprising findings
- WALL trained against Wolfpack attacks also outperforms baselines against EGA and random attacks, even though RANDOM and ROMANCE were trained against those attack types.
Limitations
- Additional computational overhead from training the planner Transformer.
- Requires hyperparameter tuning of the Wolfpack attack (m, K_WP, t_WP, T).
Experimental setup
Datasets: MPE predator-prey (PP_3/1, PP_6/2, PP_9/3) SMAC (2s3z, 3m, 3s_vs_3z, 8m, MMM, 1c3s5z)
Benchmarks: MPE predator-prey SMAC
Metrics: average cumulative reward win rate
Baselines: Vanilla QMIX RANDOM RARL RAP ERNIE ROMANCE EGA Random Attack
Builds on: QMIX RARL RAP ROMANCE ERNIE
Closest papers by meaning
- Byzantine Robust Cooperative Multi-Agent Reinforcement Learning as a Bayesian Game (ICLR 2024 · similarity 0.88)
- Robust and Diverse Multi-Agent Learning via Rational Policy Gradient (NeurIPS 2025 · similarity 0.87)
- AdvEvo-MARL: Shaping Internalized Safety through Adversarial Co-Evolution in Multi-Agent Reinforcement Learning (ICML 2026 · similarity 0.87)
- Rethinking Adversarial Policies: A Generalized Attack Formulation and Provable Defense in RL (ICLR 2024 · similarity 0.85)
- Robust Communicative Multi-Agent Reinforcement Learning with Active Defense (AAAI 2024 · similarity 0.85)
- Robust Adversarial Reinforcement Learning via Bounded Rationality Curricula (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.