Research map / Multi-agent systems and games / Cooperative multi-agent RL
Towards Efficient Collaboration via Graph Modeling in Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes f-MAT, a factor-based multi-agent transformer that models neighboring-agent cooperation as a bipartite agent-factor graph, enabling efficient message passing and parallel action decoding within CTDE.
Problem
Decentralized execution in CTDE limits coordination among agents because each acts only on local observations, while fully centralized transformer methods (MAT) are costly; graph-based agent-to-agent communication often scales as O(n^2).
Approach
Group agents into overlapping factors (hypernodes) forming a bipartite agent-factor graph, and use masked factor-based attention so messages flow only between agents and their factors; this gives complexity O(m·S_f·L) and allows parallel action generation via synchronized message passing (Gibbs-like) instead of autoregressive decoding.
Key results
- On 12x12 GridSim, MAT-dec and MAPPO reach about 75% (10x10) and 65% (12x12) of f-MAT's performance.
- f-MAT needs about half the training time of MAPPO and MAT-dec to reach similar performance on GridSim.
- On 12x12 GridSim inference, f-MAT (L_enc=1) takes 0.0096 s versus 0.2695 s for MAT, while MAPPO takes 0.0051 s.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use f-MAT instead of MAPPO or MAT-dec | GridSim 10×10 and 12×12 · GridSim | performance: 75%; 65% | Helped |
| use f-MAT instead of MAPPO or MAT-dec | GridSim 8×8 · GridSim 8×8 | training time to similar performance: 1/2 | Helped |
| use f-MAT instead of MAPPO and MAT-dec | Monaco traffic control · Monaco | training time to comparable results: 3/5 | Helped |
| use f-MAT instead of MAT-dec | traffic light control · Monaco | performance | Helped |
| use f-MAT instead of MAT | microgrid-20 · microgrid-20 | performance | Helped |
| increase L_enc from 1 toward 3 (ablation) | f-MAT · GridSim 12×12, group size 6 | reward | Helped |
| increase L_enc beyond 3 (ablation) | f-MAT · GridSim | performance | Mixed |
| increase L_enc to 4 or 5 (ablation) | f-MAT with group sizes 6 to 11 · GridSim | performance | No clear effect |
| use group size 11 instead of smaller tested group sizes (ablation) | f-MAT with L_enc=3 · GridSim | performance | Helped |
| vary the number of factors (ablation) | f-MAT · Monaco | optimal reward | Mixed |
Surprising findings
- MAT, the fully centralized upper bound, performs worst on microgrid-20 despite using all information, attributed to irrelevant information in local-control settings.
Limitations
- Factor selection is manual or random and not learned; dynamic graphs are left to future work.
- Parallel decoding relies on Gibbs-like synchronized updates that may mix slowly, mitigated by initialization.
- Comparison to communication baselines on power control is taken from reported results rather than rerun.
Experimental setup
Datasets: GridSim Monaco traffic network (SUMO) microgrid-6 and microgrid-20 power grid
Benchmarks: GridSim 8x8, 10x10, 12x12 Monaco traffic light control (28 intersections) microgrid-6 microgrid-20
Metrics: global reward training efficiency (reward vs training time) inference time average queue length
Baselines: MAPPO MAT-dec MAT CommNet DIAL ConseNet
Builds on: Multi-Agent Transformer MAPPO PPO Gibbs sampling
Closest papers by meaning
- Bridging Training and Execution via Dynamic Directed Graph-Based Communication in Cooperative Multi-Agent Systems (AAAI 2025 · similarity 0.91)
- Multi-Agent Reinforcement Learning is a Sequence Modeling Problem (NeurIPS 2022 · similarity 0.90)
- Learning Multi-Agent Communication from Graph Modeling Perspective (ICLR 2024 · similarity 0.90)
- Bayesian Ego-graph inference for Networked Multi-Agent Reinforcement Learning (NeurIPS 2025 · similarity 0.87)
- Group-Graph Policy Optimization for Long-Horizon Agentic Reinforcement Learning (NeurIPS 2026 · similarity 0.86)
- Learning Efficient and Robust Multi-Agent Communication via Graph Information Bottleneck (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.