Research map / Multi-agent systems and games / Human-AI and goal-conditioned
Grounded Answers for Multi-agent Decision-making Problem through Generative World Model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Learning before Interaction (LBI), a language-guided world model (VQ-VAE tokenizer, causal-transformer dynamics, bidirectional-transformer reward) on VisionSMAC data, used to train MARL policies whose rollouts serve as grounded answers to multi-agent decision problems.
Problem
Generative models produce sketchy or misleading answers to complex multi-agent decision-making problems because they lack trial-and-error experience; existing world models focus on single-agent settings and assume known or easily deduced rewards.
Approach
Integrate a language-conditioned interactive simulator (image-tokenized dynamics model plus a bidirectional-transformer reward model learned by IRL-style likelihood maximization over expert trajectories with hindsight relabeling via task descriptions) into the MARL pipeline, train a joint policy inside the simulator with behavior regularization, and output the simulated image sequence as the answe
Key results
- LBI reaches 94.59% win rate on 1c3s5z and 95.96% on MMM2 versus at most 82.76% (MADT) among imitation baselines on training maps.
- On unseen tasks LBI achieves 56.47% on 1c3s vs 16.21% for MADT and 97.85% on 6m vs 49.28% for MADT.
- Dynamics residual term reduces state prediction error from 0.434 (wo-RT) to 0.016.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LBI instead of reward-free imitation baselines | multi-agent policies on SMAC training maps · SMAC training maps | test win rate (%) | Helped |
| Use LBI instead of offline MARL baselines | SMAC policies · SMAC maps in offline MARL comparison | test return | Helped |
| Use LBI for zero-shot transfer rather than MADT or MA-TREX | policies on unseen SMAC tasks · unseen tasks | test win rate (%) | Helped |
| Add the dynamics residual term (ablation) | LBI dynamics model · dynamics ablation maps | prediction error: 0.434 $\pm$ 0.351 -> 0.016 $\pm$ 0.023 | Helped |
| Add the dynamics residual term (ablation) | LBI · dynamics ablation maps | return (all): 14.25 $\pm$ 1.84 -> 18.91 $\pm$ 1.33 | Helped |
| Replace predicted state reference with ground-truth images (ablation) | LBI dynamics model · dynamics ablation maps | prediction error: 0.016 $\pm$ 0.023 -> 0.014 $\pm$ 0.018 | No clear effect |
| Remove image reference (ablation) | LBI dynamics model · dynamics ablation maps | prediction error: 0.016 $\pm$ 0.023 -> 0.029 $\pm$ 0.041 | Hurt |
| Remove image reference (ablation) | LBI · dynamics ablation maps | return (all): 18.91 $\pm$ 1.33 -> 18.63 $\pm$ 1.01 | Hurt |
| Remove both residual term and image reference (ablation) | LBI dynamics model · dynamics ablation maps | prediction error: 0.016 $\pm$ 0.023 -> 0.744 $\pm$ 1.164 | Hurt |
| Remove both residual term and image reference (ablation) | LBI · dynamics ablation maps | return (all): 18.91 $\pm$ 1.33 -> 12.13 $\pm$ 2.33 | Hurt |
| Add learned conditioned rewards rather than use SMAC ground-truth rewards (ablation) | LBI reward model · reward ablation training tasks | return (training): 16.68 $\pm$ 1.55 -> 19.47$\pm$ 0.77 | Helped |
| Add learned conditioned rewards rather than use SMAC ground-truth rewards (ablation) | LBI reward model · reward ablation unseen tasks | return (unseen): 14.07 $\pm$ 2.79 -> 18.54 $\pm$ 1.49 | Helped |
Surprising findings
- Ground-truth image as reference for state prediction gave no gain over the learned image reference (LBI-GTI 18.98 vs LBI 18.91 return).
- Using ground-truth SMAC rewards in the same framework (LBI-GTR) performed worse than the learned reward (16.68 vs 19.47 training return).
Limitations
- Performance depends on simulator accuracy and multi-agent policy convergence; limited out-of-domain generalization to unseen unit types.
- Generating an answer still requires considerable waiting time for MARL policy convergence, unsuitable for real-time chatbot use.
- Evaluated only on StarCraft II (SMAC).
Experimental setup
Datasets: SMAC trajectories collected with EMC and IIE
Benchmarks: StarCraft Multi-Agent Challenge
Metrics: test win rate test return prediction error
Baselines: BC MA-AIRL MADT MAPT MA-TREX BCQ-MA CQL-MA ICQ OMAR OMIGA
Builds on: VQ-VAE Decision Transformer MA-SAC TREX MaxEnt IRL
Closest papers by meaning
- Revisiting Multi-Agent World Modeling from a Diffusion-Inspired Perspective (NeurIPS 2025 · similarity 0.89)
- $\texttt{Multi}^2$: Hierarchical Multi-Agent Decision-Making with LLM-Based Agents in Interactive Environments (ICML 2026 · similarity 0.88)
- Learning to Cooperate with Humans using Generative Agents (NeurIPS 2024 · similarity 0.88)
- Acting Beyond Learning: Imagination-Assisted Decision-Making in the Visual-based Multi-Agent Cooperative Scenarios (AAAI 2025 · similarity 0.87)
- VAGEN: Reinforcing World Model Reasoning for Multi-Turn VLM Agents (NeurIPS 2025 · similarity 0.87)
- GenRL: Multimodal-foundation world models for generalization in embodied agents (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.