Research map / Multi-agent systems and games / Human-AI and goal-conditioned
AdaSociety: An Adaptive Environment with Social Structures for Multi-Agent Decision-Making
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces AdaSociety, a customizable multi-agent grid environment with expanding physical state/action spaces and adaptive multi-layer social structures, with three mini-games benchmarked on RL (PPO, RecPPO, MAPPO, Rainbow, curriculum learning) and GPT-4 LLM-based control.
Problem
Existing multi-agent environments lack a combination of expanding physical surroundings and adaptive social connections, which limits research on learning intelligent social behaviors in multi-agent decision-making.
Approach
A multi-agent environment where agents' physical actions unlock new resources/events (expanding state and action spaces) and social actions rewire a multi-layer directed social graph, which in turn changes observations and reward sharing; formalized as Growing-MG.
Key results
- In Contract-Easy, curriculum learning (CL) reaches 0.9136 of Oracle reward versus 0.2286 for PPO and 0.1987 for Rainbow.
- LLM-C (GPT-4) achieves 0.8433 normalized reward in Contract-Easy and 0.8733 in Negotiation-Easy, nearly surpassing RL baselines but not reaching Oracle.
- In Negotiation-Hard, CL achieves 0.1945 normalized reward versus 0.1093 for PPO, and RL methods fail to complete TorchCraft.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use curriculum learning with staged reward sharing, varied social structures, and agent social actions instead of general RL training | RecPPO agents · Contract Easy | average individual reward (normalized by Oracle reward): 0.2286 -> 0.9136 | Helped |
| Use curriculum learning with staged reward sharing, varied social structures, and agent social actions instead of general RL training | RecPPO agents · Contract Hard | average individual reward (normalized by Oracle reward): 0.1151 -> 0.2773 | Helped |
| Use curriculum learning with staged reward sharing, varied social structures, and agent social actions instead of general RL training | RecPPO agents · Negotiation Easy | average individual reward (normalized by Oracle reward): 0.2276 -> 0.3543 | Helped |
| Use curriculum learning with staged reward sharing, varied social structures, and agent social actions instead of general RL training | RecPPO agents · Negotiation Hard | average individual reward (normalized by Oracle reward): 0.1093 -> 0.1945 | Helped |
| Use LLM-C instead of CL | Contract Easy agents · Contract Easy | average reward: 0.9136 -> 0.8433 | Hurt |
| Change social structure dynamically from Inequality to Independent group to Overlapping group | PPO-trained policies · Social Structure Dynamic | individual reward per step | Mixed |
| Use curriculum learning instead of PPO, RecPPO, or MAPPO | agents in Social Structure Dynamic · Social Structure Dynamic | individual reward | Helped |
| Use MAPPO instead of PPO or RecPPO | agents in Social Structure · Social Structure | individual reward | Hurt |
| Use Rainbow instead of PPO or RecPPO | agents in Social Structure · Social Structure | individual reward | Hurt |
| Train with agent groups rather than without agent groups | current algorithms · Social Structure | policy stability | Hurt |
| Use LLM-C instead of RL-based methods | agents in Social Structure Hard · Social Structure Hard | average reward | Helped |
Surprising findings
- Larger contract groups do not correlate with higher returns; Rainbow forms large groups but earns low rewards.
- LLM-C underperforms the curriculum learning method on Contract-Easy despite commonsense reasoning.
Limitations
- No human interface; human-machine interaction not yet supported.
- Current RL and LLM algorithms struggle to exploit social structures, with high variance in group-based scenarios.
- Survival pressures and other negative incentives not yet included.
Experimental setup
Datasets: AdaSociety mini-games (Social Structure, Contract, Negotiation, Exploration)
Benchmarks: AdaSociety Social Structure AdaSociety Contract AdaSociety Negotiation
Metrics: individual reward fairness score completion rate average degree maximum degree normalized reward vs Oracle
Baselines: PPO RecPPO MAPPO Rainbow Curriculum learning LLM-C (GPT-4 + rule-based controller) Random
Builds on: Markov Game PPO Rainbow MAPPO Unsupervised Environment Design
Closest papers by meaning
- Efficient Adaptation in Mixed-Motive Environments via Hierarchical Opponent Modeling and Planning (ICML 2024 · similarity 0.86)
- ALSO: Adversarial Online Strategy Optimization for Social Agents (ICML 2026 · similarity 0.86)
- Melting Pot Contest: Charting the Future of Generalized Cooperative Intelligence (NeurIPS 2024 · similarity 0.86)
- LLM-Assisted Semantically Diverse Teammate Generation for Efficient Multi-agent Coordination (ICML 2025 · similarity 0.86)
- $\texttt{Multi}^2$: Hierarchical Multi-Agent Decision-Making with LLM-Based Agents in Interactive Environments (ICML 2026 · similarity 0.86)
- Mental-Models for Multi-Agent Systems (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.