Research map / Multi-agent systems and games / Cooperative multi-agent RL
HGAP: Boosting Permutation Invariant and Permutation Equivariant in Multi-Agent Reinforcement Learning via Graph Attention Network
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HGAP, a GATv2-based agent network for MARL that enforces permutation invariance and equivariance and handles POMDP by inferring unseen entities through the entity graph, evaluated on SMAC, SMACv2, and MPE.
Problem
Conventional MARL agent networks often ignore permutation invariance (PI) and permutation equivariance (PE) of observed entities, which slows training and hurts transferability, and POMDP limits the number of observable entities.
Approach
Treat observed entities as a graph and use a GATv2-based entity relation module with hypernetwork-generated transformations, aggregating entity embeddings order-independently for ego actions and using pairwise entity-action scoring for engagement actions, so PI and PE hold by design while graph message passing lets the agent infer out-of-sight entities.
Key results
- In SMAC, HGAP reaches 100% test win rate on many hard and super-hard maps (e.g., 3s5z, 6h vs 8z, 27m vs 30m) where HPN and other baselines fall below.
- HGAP transfer from pretrained scenarios reduces training time by at least 60% compared with training from scratch.
- Integrating HGAP into QPLEX and MAPPO improves test win rates and training efficiency on SMACv2 scenarios.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use HGAP instead of HPN | QMIX agent network · SMAC hard and super-hard scenarios, including 5m vs 6m, 3s5z vs 3s6z, and MMM2 | test win rate: 98 -> 100; 97 -> 100; 97 -> 100 | Helped |
| Use HGAP instead of HPN | QMIX agent network · SMACv2, protoss 20 vs 23 | test win rate: 92 -> 100 | Helped |
| Use HGAP instead of HPN | QMIX agent network · SMACv2, terran 10 vs 11 | test win rate: 89 -> 96 | Helped |
| Use HGAP instead of HPN | QMIX agent network · SMACv2, zerg 5 vs 5 | test win rate: 94 -> 92 | Hurt |
| Add HGAP to MADDPG (ablation) | MADDPG on MPE · Navigation (N=10) and Predator-Prey (N=3) | test win rate | Helped |
| Replace QMIX with QPLEX while using HGAP (ablation) | HGAP on SMACv2 · SMACv2 scenarios | test win rate | Helped |
| Add HGAP to MAPPO (ablation) | MAPPO on SMACv2 · SMACv2 scenarios | test win rate | Helped |
| Add HGAP to an agent network (ablation) | QMIX-based MARL · SMAC PI/PE comparison scenarios | test win rate | Helped |
| Pretrain HGAP in a source scenario before transfer (ablation) | HGAP · SMAC transfer scenarios | training time: at least a 60% reduction | Helped |
| Use HGAP rather than training from scratch (ablation) | HGAP policy in transfer learning · SMAC target scenarios | training time: at least a 60% reduction | Helped |
| Use graph-based HGAP under opponents with full environmental information (ablation) | HGAP · SMAC 3s5z and corridor | test win rate | Helped |
| Increase the entity population while using HGAP (ablation) | HGAP-MADDPG · MPE Navigation (N=100) and Predator-Prey (N=200) | test win rate | Helped |
Surprising findings
- Attention-based UPDeT shows inconsistent attention on hard scenarios, so attention alone does not guarantee useful entity correlations.
Limitations
- Paper notes absence of coach-based or human-in-the-loop methods to enhance learning.
- Experiments limited to MPE, SMAC, and SMACv2 simulated environments.
Experimental setup
Datasets: SMAC SMACv2 MPE
Benchmarks: SMAC SMACv2 MPE
Metrics: test win rate
Baselines: RNN UPDeT DFAC ResQ RODE ASN HPN DA-MADDPG Set Transformer GNN
Builds on: GATv2 QMIX QPLEX UPDeT HPN
Closest papers by meaning
- Boosting Multiagent Reinforcement Learning via Permutation Invariant and Permutation Equivariant Networks (ICLR 2023 · similarity 0.91)
- Boosting Sample Efficiency and Generalization in Multi-agent Reinforcement Learning via Equivariance (NeurIPS 2024 · similarity 0.87)
- HYGMA: Hypergraph Coordination Networks with Dynamic Grouping for Multi-Agent Reinforcement Learning (ICML 2025 · similarity 0.86)
- SiT: Symmetry-invariant Transformers for Generalisation in Reinforcement Learning (ICML 2024 · similarity 0.85)
- Investigating Relational State Abstraction in Collaborative MARL (AAAI 2025 · similarity 0.84)
- Bridging Training and Execution via Dynamic Directed Graph-Based Communication in Cooperative Multi-Agent Systems (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.