Research map / Multi-agent systems and games / Games and equilibria
Aligning Individual and Collective Objectives in Multi-Agent Cooperation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Altruistic Gradient Adjustment (AgA), a gradient-based optimizer for mixed-motive cooperation in differentiable games that pulls learning dynamics toward stable fixed points of the collective objective while accounting for individual interests, and introduces the Selfish-MMM2 StarCraft II testbed.
Problem
Mixed-motive cooperation in multi-agent learning, where individual and collective objectives are misaligned; existing fixes rely on hand-crafted reward design or lack theoretical grounding for convergence.
Approach
Model mixed-motive games as differentiable games and adjust the collective gradient with a term combining individual gradients and the collective Hessian-vector product, with a sign of the alignment parameter chosen so the update is pulled toward stable fixed points of the collective loss (and pushed from unstable ones) near fixed points.
Key results
- In the two-player public goods game, AgA reaches social welfare 2.903 versus 2.814 for Simul-Co and equality 0.960, the highest among methods.
- On Selfish-MMM2, AgA attains a 92.64% win rate against the built-in SMAC bot, 23.61 points above the next best (SL).
- In Cleanup, AgA averages social welfare 105.15, about 56% above the runner-up SL; in Harvest AgA improves on SVO by 33.55 average social welfare.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use AgA instead of Simul-Co | two-player public goods game · Two-player public goods game | social welfare (SW): 2.814 -> 2.903 | Helped |
| use AgA instead of Simul-Co | two-player public goods game · Two-player public goods game | equality (E): 0.941 -> 0.960 | Helped |
| use AgA (λ=1) instead of Simul-Co | Harvest · Harvest | equality (E): 0.975 -> 0.988 | Helped |
| use AgA (λ=100) instead of Simul-Co | Cleanup · Cleanup | equality (E): 0.948 -> 0.959 | Helped |
| use AgA instead of the next-best method, SVO | Harvest · Harvest | social welfare (SW): +33.55 | Helped |
| use AgA instead of the runner-up, SL | Cleanup · Cleanup | social welfare (SW): approximately a 56% increase | Helped |
| use AgA instead of the next-best method, SL | Selfish-MMM2 · Selfish-MMM2 | win rate: 23.61 percent more | Helped |
| add sign alignment to AgA (ablation) | AgA · toy two-player differentiable mixed-motive game | steps to complete trajectory: approximately 15% | Helped |
| use λ=100 instead of λ=0.1 (ablation) | AgA · Harvest | equality (E): 0.981 ± 0.006 -> 0.982 ± 0.012 | No clear effect |
| use λ=100 instead of λ=1 (ablation) | AgA · Cleanup | equality (E): 0.956 ± 0.007 -> 0.959 ± 0.011 | No clear effect |
| use λ=1000 instead of λ=100 (ablation) | AgA · Cleanup | equality (E): 0.959 ± 0.011 -> 0.905 ± 0.022 | Hurt |
| add sign alignment instead of using AgA-Sign (ablation) | AgA · Cleanup and Harvest | social welfare (SW) | Helped |
Surprising findings
- Simul-Co, which ignores individual interests, is nearly as strong as AgA on social welfare in the public goods game, and AgA is slower to converge than Simul-Co in that setting.
- AgA has a per-step cost about 2.7 times that of Simul-Ind, so its advantage depends on fewer total steps needed to converge.
Limitations
- Focuses on converging to stable fixed points of the collective objective rather than deeply modeling individual objective dynamics.
- Evaluated environments remain removed from real-world mixed-motive applications.
- Computational cost of Hessian-vector products makes AgA roughly 2-3 times slower per step than gradient-based baselines.
Experimental setup
Datasets: Cleanup Harvest SMAC MMM2 map (adapted as Selfish-MMM2) two-player public goods game
Benchmarks: two-player public goods game Cleanup Harvest Selfish-MMM2
Metrics: social welfare equality (Gini-based) individual rewards win rate
Baselines: Simul-Ind Simul-Co CGA SGA SVO SL (selfishness level) AgA-Sign
Builds on: consensus optimization (CGA) symplectic gradient adjustment (SGA) IPPO MAPPO social value orientation
Closest papers by meaning
- Fair Cooperation in Mixed-Motive Games via Conflict-Aware Gradient Adjustment (NeurIPS 2025 · similarity 0.92)
- Learning to Balance Altruism and Self-interest Based on Empathy in Mixed-Motive Games (NeurIPS 2024 · similarity 0.88)
- Beyond Mandatory Federations: Balancing Egoism, Utilitarianism and Egalitarianism in Mixed-Motive Games (AAAI 2025 · similarity 0.88)
- Mastering Zero-Shot Interactions in Cooperative and Competitive Simultaneous Games (ICML 2024 · similarity 0.87)
- Training Generalizable Collaborative Agents via Strategic Risk Aversion (NeurIPS 2026 · similarity 0.87)
- Efficient Adaptation in Mixed-Motive Environments via Hierarchical Opponent Modeling and Planning (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.