Research map / Multi-agent systems and games / Games and equilibria

Training Generalizable Collaborative Agents via Strategic Risk Aversion

NeurIPS 2026 · Multi-agent systems and games › Games and equilibria · approach: Cooperation and social dilemmas · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes strategically risk-averse policy optimization (SRPO), an IPPO-style MARL method that trains each agent against a KL-constrained adversarial partner, to get collaborators that avoid free-riding and generalize to unseen partners.

Problem

Collaborative multi-agent learning produces policies that fail to generalize to new partners, partly because agents free-ride on partners' effort and overfit to conventions.

Approach

Uses risk-averse quantal response equilibrium (entropic risk with KL-constrained worst-case partner) as an inductive bias; implements it as an adversary-augmented policy optimization objective that adds minimal changes to PPO, and proves risk can increase shared reward and eliminate free-riding in structured games.</µcore_idea>

Key results

• On multi-LLM debate GSM8K, SRPO improves cross-play joint accuracy over IPPO in all six Qwen pairings, with gains up to 19.27% (Q0.6B&Q4B: 0.848 vs 0.711).
• When paired with an untuned Llama-3.2-1B-Instruct, SRPO-trained agents improve individual accuracy over IPPO by up to 14.49% (Q3B: 0.632 vs 0.552).
• In Overcooked, IPPO converges to free-riding checkerboard cross-play patterns while SRPO (tau=10, epsilon=0.1) yields both agents contributing and higher cross-play performance.

Surprising findings

• In Tag, SRPO has slightly lower training-environment performance than IPPO, showing risk aversion can trade off in-distribution performance.
• Entropy regularization alone (IPPO) does not eliminate free-riding in the reported experiments.

Limitations

• Theoretical free-lunch results proven only for structured games (quadratic aggregative, two-player symmetric finite games).
• LLM experiment is described as a small-scale proof of concept.
• Code to be released only upon publication.

Experimental setup

Datasets: GSM8K

Benchmarks: Overcooked gridworld Tag Hanabi GSM8K

Metrics: cross-play reward joint accuracy individual accuracy

Baselines: IPPO

Builds on: IPPO PPO quantal response equilibrium risk-averse quantal response equilibrium

Closest papers by meaning

• Diversity Is Not All You Need: Training A Robust Cooperative Agent Needs Specialist Partners (NeurIPS 2024 · similarity 0.89)
• Cross-environment Cooperation Enables Zero-shot Multi-agent Coordination (ICML 2025 · similarity 0.89)
• Adaptively Coordinating with Novel Partners via Learned Latent Strategies (NeurIPS 2025 · similarity 0.88)
• Cautiously-Optimistic Knowledge Sharing for Cooperative Multi-Agent Reinforcement Learning (AAAI 2024 · similarity 0.88)
• Multi-agent cooperation through learning-aware policy gradients (ICLR 2025 · similarity 0.88)
• Collaborative Mean Estimation Among Heterogeneous Strategic Agents: Individual Rationality, Fairness, and Truthful Contribution (ICML 2025 · similarity 0.88)