Research map / Multi-agent systems and games / Cooperative multi-agent RL
Bayesian Ego-graph inference for Networked Multi-Agent Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BayesG, a decentralized actor-critic for networked MARL that samples per-agent ego-graph communication masks via variational (ELBO) inference, evaluated on SUMO traffic signal control benchmarks up to 167 agents.
Problem
Networked multi-agent RL under local observability and fixed physical graphs, where static communication neighborhoods ignore contextual relevance and centralized methods require global state access.
Approach
Each agent learns a latent binary communication mask over its physical ego-graph using variational Bayesian inference, trained end-to-end with the actor loss as the likelihood term in an ELBO, so the policy and interaction topology are optimized jointly without global supervision.
Key results
- BayesG steadily outperforms IA2C, ConseNet, FPrint, LToS, CommNet and NeurComm across all five ATSC environments in training return over 5 seeds.
- On large-scale NewYork33/51/167 (up to 167 agents), BayesG reaches higher asymptotic returns with more stable learning curves than baselines.
- Ablations on Monaco and NewYork51 show random masking severely degrades performance while the learned mask performs best; trajectory features outperform state-only mask inputs.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use BayesG's learned variational ego-graph mask instead of fixed communication | BayesG · ATSC benchmarks | episode return | Helped |
| Replace the learned mask with random masking (ablation) | BayesG · Monaco and NewYork51 | performance | Hurt |
| Use a learned variational mask instead of no masking (ablation) | BayesG · Monaco and NewYork51 | performance | Helped |
| Generate the mask from trajectory features rather than state-only features (ablation) | BayesG · Monaco and NewYork51 | performance | Helped |
| Generate the mask from policy features (ablation) | BayesG · Monaco and NewYork51 | performance | Helped |
| Combine state, trajectory, and policy features to generate the mask (ablation) | BayesG · Monaco and NewYork51 | performance | Helped |
Surprising findings
- Random edge masking severely degrades performance relative to no masking, indicating naive edge dropout is harmful.
Limitations
- Gains are shown only on traffic signal control; generality to other networked MARL domains is untested.
- Mask prior is a simple Bernoulli with a retention bias, and the likelihood is approximated by the actor loss rather than a true data likelihood.
- Experiments use a single 5-seed setup with reported results primarily as training curves rather than a comprehensive test-time table.
Experimental setup
Datasets: ATSC_Grid Monaco NewYork33 NewYork51 NewYork167
Benchmarks: ATSC_Grid Monaco NewYork33 NewYork51 NewYork167
Metrics: episode return (negative total queue length) training reward convergence
Baselines: IA2C ConseNet FPrint LToS CommNet NeurComm
Builds on: soft actor-critic gumbel-softmax A2C NeurComm CommNet
Closest papers by meaning
- Context-Aware Bayesian Network Actor-Critic Methods for Cooperative Multi-Agent Reinforcement Learning (ICML 2023 · similarity 0.91)
- Graph Diffusion for Robust Multi-Agent Coordination (ICML 2025 · similarity 0.89)
- Bridging Training and Execution via Dynamic Directed Graph-Based Communication in Cooperative Multi-Agent Systems (AAAI 2025 · similarity 0.88)
- Causality Meets Locality: Provably Generalizable and Scalable Policy Learning for Networked Systems (NeurIPS 2025 · similarity 0.87)
- Group-Graph Policy Optimization for Long-Horizon Agentic Reinforcement Learning (NeurIPS 2026 · similarity 0.87)
- Towards Efficient Collaboration via Graph Modeling in Reinforcement Learning (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.