Research map / Multi-agent systems and games / Cooperative multi-agent RL
Byzantine Robust Cooperative Multi-Agent Reinforcement Learning as a Bayesian Game
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Frames Byzantine attacks in cooperative MARL as a Bayesian adversarial robust Dec-POMDP and proposes an ex interim robust equilibrium solved by a two-timescale actor-critic (EIR-MAPPO), evaluated on matrix game, LBF and SMAC.
Problem
Cooperative MARL policies become non-robust when allies fail arbitrarily (Byzantine failure) or are compromised by a non-oblivious adversary, and agents do not know which allies are adversarial.
Approach
Model Byzantine agents as nature-chosen types in a Bayesian game (BARDec-POMDP); agents act on posterior beliefs over types (ex interim equilibrium) rather than the prior (ex ante), with a robust Harsanyi-Bellman equation and a two-timescale actor-critic with belief network.
Key results
- EIR-MAPPO surpasses baselines by 5.81% on Toy, 5.88% on LBF, and 25.45% on SMAC averaged across uncertainties.
- Under non-oblivious worst-case attacks, EIR-MAPPO approaches the True Type ideal defense on all three environments while keeping cooperative performance on par with MAPPO.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use EIR-MAPPO with posterior beliefs instead of ex ante robust MAPPO without belief (ablation) | MAPPO · Toy | average reward under diverse uncertainties: 5.81% | Helped |
| Use EIR-MAPPO with posterior beliefs instead of ex ante robust MAPPO without belief (ablation) | MAPPO · LBF | average reward under diverse uncertainties: 5.88% | Helped |
| Use EIR-MAPPO with posterior beliefs instead of ex ante robust MAPPO without belief (ablation) | MAPPO · SMAC | average reward under diverse uncertainties: 25.45% | Helped |
| Use EIR-MAPPO instead of MAPPO | MAPPO · Toy, LBF, and SMAC under non-oblivious attacks | robust performance | Helped |
| Use EIR-MAPPO instead of MAPPO | MAPPO · Toy, LBF, and SMAC without attacks | cooperative performance | No clear effect |
| Use EIR-MAPPO instead of RMAAC | RMAAC · Observation perturbations | robustness | No clear effect |
| Use EIR-MAPPO instead of RMAAC | RMAAC · Unseen action perturbations | robustness | Helped |
Surprising findings
- Pure-strategy equilibria are not guaranteed in robust c-MARL, so optimal robust policies are stochastic.
Limitations
- Convergence guarantee is asymptotic and relies on stochastic approximation assumptions; no finite-sample or global guarantees.
- Threat model assumes a single attacker/adversarial agent per episode.
- Ex ante vs ex interim value gap is not established in general.
Experimental setup
Datasets: toy iterative matrix game Level-based Foraging 12x12-4p-3f-c SMAC 4m vs 3m
Benchmarks: toy iterative matrix game Level-based Foraging 12x12-4p-3f-c SMAC 4m vs 3m
Metrics: episode reward under attack average reward across uncertainties
Baselines: MADDPG M3DDPG MAPPO RMAAC EAR-MAPPO
Builds on: MAPPO M3DDPG RMAAC Harsanyi-Bellman ad hoc coordination robust MDP
Closest papers by meaning
- Robust Multi-Agent Reinforcement Learning with Stochastic Adversary (ICML 2025 · similarity 0.89)
- Empirical Study on Robustness and Resilience in Cooperative Multi-Agent Reinforcement Learning (NeurIPS 2025 · similarity 0.89)
- Breaking the Curse of Multiagency in Robust Multi-Agent Reinforcement Learning (ICML 2025 · similarity 0.89)
- Robust and Diverse Multi-Agent Learning via Rational Policy Gradient (NeurIPS 2025 · similarity 0.89)
- Wolfpack Adversarial Attack for Robust Multi-Agent Reinforcement Learning (ICML 2025 · similarity 0.88)
- Game-Theoretic Robust Reinforcement Learning Handles Temporally-Coupled Perturbations (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.