Research map / Multi-agent systems and games / Cooperative multi-agent RL
Leveraging Partial Symmetry for Multi-Agent Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces partially symmetric Markov games and the PSE framework, which adaptively exploits symmetry (augmentation and consistency loss) in MARL under symmetry breaking, with a bounded performance error proof.
Problem
Standard MARL symmetry-exploitation methods assume perfect permutation invariance or spatial symmetry, which rarely holds in real-world multi-agent tasks where non-uniform fields or heterogeneity break symmetry.
Approach
Defines partial equivariance/invariance and partially symmetric Markov games (reward and transition deviations bounded by epsilon and MMD delta), proves the Q-function error from symmetry-augmented data is bounded by (epsilon + gamma*delta)/(1-gamma), and adaptively weights symmetry via a quantified symmetry degree D with exponential annealing applied to augmentation probability and a symmetry con
Key results
- PSE improved MAPPO, QMIX and MADDPG over their original versions across Predator-Prey, Cooperative Navigation and Formation Change, with 10 random seeds.
- On real-world E-puck formation change, the rate of risky states was 2.1% for MAPPO-PSE versus 5.6% for MAPPO.
- In Wildlife Monitoring under symmetry breaking, EQ-MPN degraded below classic MPN while MPN-PSE continued to improve.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply PSE to MAPPO | MAPPO · Real-world formation change | rate of risky states: 5.6% -> 2.1% | Helped |
| Apply PSE to MADDPG, QMIX, and MAPPO | MARL algorithms · Predator-Prey | evaluation reward | Helped |
| Apply PSE to MADDPG, QMIX, and MAPPO | MARL algorithms · Cooperative Navigation | performance | Helped |
| Apply PSE to MAPPO | MAPPO · Formation Change | reward | Helped |
| Use PSE with MPN instead of plain MPN or EQ-MPN | MPN · Wildlife Monitoring, partial symmetry | final convergence quality | Helped |
| Replace perfect-symmetry EQ-MPN with PSE-based MPN | MPN · Wildlife Monitoring, partial symmetry | performance | Helped |
| Add symmetry augmentation and symmetry loss with a fixed coefficient of 0.5, without PSE's adaptive framework (ablation) | MAPPO · Tasks with increasing noise intensity | performance | Hurt |
| Use full PSE instead of MAPPO-SE with a fixed symmetry-loss coefficient (ablation) | MAPPO · Different degrees of symmetry-breaking | performance | Helped |
Surprising findings
- The perfect-symmetry method MAPPO-SE performed worse than plain MAPPO as noise intensity increased.
- EQ-MPN, which embeds perfect symmetry, fell below the classic MPN under partial symmetry.
Limitations
- Symmetry-breaking is simulated by injected noise in transitions rather than natural asymmetries
- Extension to heterogeneous agents is left to future work
- Real-world evaluation is limited to a small robot swarm with a single formation-change task
Experimental setup
Datasets: Predator-Prey Cooperative Navigation Wildlife Monitoring Formation Change (Webots)
Benchmarks: Predator-Prey Cooperative Navigation Wildlife Monitoring Formation Change
Metrics: evaluation reward convergence speed risky state rate
Baselines: MADDPG QMIX MAPPO MPN EQ-MPN MAPPO-SE
Builds on: MAPPO MADDPG QMIX EQ-MPN symmetric Markov game
Closest papers by meaning
- ${\rm E}(3)$-Equivariant Actor-Critic Methods for Cooperative Multi-Agent Reinforcement Learning (ICML 2024 · similarity 0.91)
- Playing Markov Games Without Observing Payoffs (NeurIPS 2026 · similarity 0.87)
- Foundations of Multi-Agent Learning in Dynamic Environments: Where Reinforcement Learning Meets Strategic Decision-Making (AAAI 2025 · similarity 0.87)
- Convex Markov Games: A New Frontier for Multi-Agent Reinforcement Learning (ICML 2025 · similarity 0.87)
- Boosting Sample Efficiency and Generalization in Multi-agent Reinforcement Learning via Equivariance (NeurIPS 2024 · similarity 0.87)
- Incentivize without Bonus: Provably Efficient Model-based Online Multi-agent RL for Markov Games (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.