Research map / Multi-agent systems and games / Games and equilibria
Fair Cooperation in Mixed-Motive Games via Conflict-Aware Gradient Adjustment
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FCGrad, a conflict-aware gradient adjustment for mixed-motive MARL that projects the gradient of the lower-valued objective onto the normal plane of the other to balance cooperation and fairness, evaluated on Unfair Coins, Cleanup, and Harvest.
Problem
Mixed-motive multi-agent RL requires balancing individual and collective returns, but reward restructuring methods (gifting, intrinsic motivation) boost collective return without addressing fairness in agents' task-specific rewards.
Approach
When individual and collective policy gradients conflict (negative inner product), project the gradient of the objective with lower value onto the normal plane of the other, so the disadvantaged objective improves without harming the other; otherwise use a weighted sum of the two gradients.
Key results
- In Cleanup, FCGrad achieves the best GeoMean and Min returns among compared methods and Mean comparable to Col.
- On the Coin game, FCGrad reaches Gini 0.010 and Jain 0.999, the most fair of compared methods.
- In Harvest, FCGrad outperforms baselines across alpha values, with Gini 0.093 and Jain 0.959.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FCGrad instead of PCGrad | Mixed-motive MARL · Unfair Coin | Gini: 0.039 -> 0.010 | Helped |
| Use FCGrad instead of PCGrad | Mixed-motive MARL · Unfair Coin | Jain's index: 0.994 -> 0.999 | Helped |
| Use FCGrad instead of Weighted | Mixed-motive MARL · Cleanup | Gini: 0.266 -> 0.223 | Helped |
| Use FCGrad instead of Weighted | Mixed-motive MARL · Cleanup | Jain's index: 0.801 -> 0.835 | Helped |
| Use FCGrad instead of IA | Mixed-motive MARL · Harvest | Gini: 0.087 -> 0.093 | Hurt |
| Use FCGrad instead of IA | Mixed-motive MARL · Harvest | Jain's index: 0.973 -> 0.959 | Hurt |
| Vary the no-conflict weighting factor beta; use a value between 0.7 and 0.8 (ablation) | FCGrad · Harvest | GeoMean | Mixed |
Surprising findings
- Ignoring the collective objective (pure individual reward optimization) fails to learn effectively in Cleanup and Harvest.
Limitations
- Theoretical convergence guarantee requires gradient-conflict recurrence, which may not hold in practice and depends on the weighting factor beta.
- Guarantees assume L-smoothness and Robbins-Monro step sizes; empirical validation limited to small grid environments with 4 agents (2 agents in Coins).
Experimental setup
Datasets: Unfair Coin Game (Coins) Cleanup Harvest
Benchmarks: Unfair Coins Cleanup Harvest
Metrics: alpha-fairness return (Mean, GeoMean, Min) Gini coefficient Jain's index
Baselines: Col Ind IA Weighted PCGrad AgA
Builds on: PCGrad Altruistic Gradient Adjustment IPPO generalized advantage estimation inequity aversion
Closest papers by meaning
- Aligning Individual and Collective Objectives in Multi-Agent Cooperation (NeurIPS 2024 · similarity 0.92)
- Learning to Balance Altruism and Self-interest Based on Empathy in Mixed-Motive Games (NeurIPS 2024 · similarity 0.89)
- Multi-agent cooperation through learning-aware policy gradients (ICLR 2025 · similarity 0.89)
- Efficient Adaptation in Mixed-Motive Environments via Hierarchical Opponent Modeling and Planning (ICML 2024 · similarity 0.88)
- Reciprocal Reward Influence Encourages Cooperation From Self-Interested Agents (NeurIPS 2024 · similarity 0.87)
- Beyond Mandatory Federations: Balancing Egoism, Utilitarianism and Egalitarianism in Mixed-Motive Games (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.