Research map / Multi-agent systems and games / Cooperative multi-agent RL
Inverse Attention Agents for Multi-Agent Systems
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Inverse Attention Agents that infer other same-type agents' goal-attention weights with an inverse attention network and use them to update their own attention, improving ad-hoc cooperation and competition in MPE-based environments.
Problem
Multi-agent systems where agents must adapt to unseen teammates and opponents; agents trained with conventional MARL excel only with their training cohorts and degrade with unfamiliar agents.
Approach
Attention over gradient-field goals is used as an explicit mental state; an inverse attention network trained on the agent's own attention data infers teammates' attention from their observations and past actions, and these inferred weights are fused with the agent's own attention weights before producing actions.
Key results
- Inverse-Att achieves 404.14 on Spread vs 283.89 for Self-Att and 31.82 for MAPPO in the mix-and-match evaluation.
- In human experiments, Inverse-Att scores 332.3 on Spread versus 272.0 for Self-Att and 16.8 for MAPPO.
- Top-attention prediction accuracy of the inverse attention network reaches nearly 100%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add inverse attention inference and use inferred teammate attention to update policy attention weights (ablation) | Self-Att · Spread | average reward: 283.89±6.23 -> 404.14±5.40 | Helped |
| add inverse attention inference and use inferred teammate attention to update policy attention weights (ablation) | Self-Att · Adversary Sheep | average reward: -23.15±0.83 -> -16.91±1.84 | Helped |
| add inverse attention inference and use inferred teammate attention to update policy attention weights (ablation) | Self-Att · Adversary Wolf | average reward: 107.93±2.26 -> 110.15±3.74 | Helped |
| add inverse attention inference and use inferred teammate attention to update policy attention weights (ablation) | Self-Att · Grassland Sheep | average reward: -10.02±5.19 -> 28.59±0.93 | Helped |
| add inverse attention inference and use inferred teammate attention to update policy attention weights (ablation) | Self-Att · Grassland Wolf | average reward: 93.68±1.61 -> 101.21±2.84 | Helped |
| add inverse attention inference and use inferred teammate attention to update policy attention weights (ablation) | Self-Att · Navigation | average reward: 328.24±7.76 -> 497.96±10.75 | Helped |
| add inverse attention inference and use inferred teammate attention to update policy attention weights (ablation) | Self-Att · Tag Sheep | average reward: -11.74±0.65 -> -5.99±0.44 | Helped |
| add inverse attention inference and use inferred teammate attention to update policy attention weights (ablation) | Self-Att · Tag Wolf | average reward: 67.60±1.00 -> 109.81±1.36 | Helped |
| replace gradient-field inputs with raw environmental observations (ablation) | Inverse-Att · Spread | average reward: 348.66 -> 404.14 | Hurt |
| replace gradient-field inputs with raw environmental observations (ablation) | Inverse-Att · Adversary Sheep | average reward: -19.42 -> -16.91 | Hurt |
| replace gradient-field inputs with raw environmental observations (ablation) | Inverse-Att · Adversary Wolf | average reward: 95.17 -> 110.15 | Hurt |
| replace gradient-field inputs with raw environmental observations (ablation) | Inverse-Att · Grassland Sheep | average reward: 21.50 -> 28.59 | Hurt |
Surprising findings
- Self-Att agents and Inverse-Att agents both outperformed human participants in the tested environments.
- Inverse-Att-Vanilla (raw relative positions instead of gradient fields) does not beat Inverse-Att on Grassland wolf (103.05 vs 101.21 favors vanilla).
Limitations
- Attention inference is restricted to agents of the same type.
- Attention model for the UW network does not accommodate arbitrary numbers of inferred attention weights.
- Human experiments involve only five participants.
Experimental setup
Datasets: MPE-based environments (Spread, Adversary, Grassland, Navigation, Tag) synthetic gradient field training data
Benchmarks: Spread Adversary Grassland Navigation Tag human experiments
Metrics: average episode reward attention prediction accuracy
Baselines: MAPPO IPPO MAA2C ToM2C* Self-Att
Builds on: MAPPO Denoised Score Matching ToM2C Multi-agent Particle Environment
Closest papers by meaning
- Estimating cognitive biases with attention-aware inverse planning (NeurIPS 2025 · similarity 0.87)
- Adaptive Theory of Mind for LLM-based Multi-Agent Coordination (AAAI 2026 · similarity 0.87)
- Mental-Models for Multi-Agent Systems (NeurIPS 2026 · similarity 0.87)
- Adaptively Coordinating with Novel Partners via Learned Latent Strategies (NeurIPS 2025 · similarity 0.85)
- NestRL: A Nested Training Regime for Mutual Adaptation in Human–AI Teaming (NeurIPS 2026 · similarity 0.85)
- A Theoretical Study of (Hyper) Self-Attention through the Lens of Interactions: Representation, Training, Generalization (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.