Research map / Multi-agent systems and games / Cooperative multi-agent RL

World Models as Adversaries: Multi-Agent Self-Play Fine-Tuning for Robust Motion Planning

NeurIPS 2026 · Multi-agent systems and games › Cooperative multi-agent RL · approach: MARL benchmarks and grouping · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Adversarial World Modeling (AWM), a self-play fine-tuning framework where a role-conditioned autoregressive world model acts as an adversary (sparse coalitions, counterfactual credit) and the planner is trained via regret-CVaR with trust-region retention on nuPlan and InterPlan.

Problem

Autoregressive motion planners trained on naturalistic logs are brittle in rare, safety-critical interactive scenarios; existing adversarial training relies on external attackers, heuristics, or simulator-heavy rollouts that do not fit tokenized planners.

Approach

Convert the planner's own predictive world model into a role-conditioned adversary that selects sparse attack coalitions (primary plus calibrated partner) and train the ego planner as a constrained regret-CVaR best response against this frozen adversary, in a decoupled two-stage solver.

Key results

• AWM-Planner improves Plan-R1 on InterPlan by +2.10, InterPlan10 by +4.97, and InterPlan-LongTail by +5.81.
• On Test14-hard NR, AWM-Planner scores 80.05 vs Plan-R1 77.45 (+2.60).
• Used as simulator-side adversary, AWM reduces mean sim-agent R-Score across seven planners from 70.24 to 67.46.

Surprising findings

• Simultaneous alternating self-play is less stable (2/3 stable seeds) and costlier than the decoupled pipeline.
• Removing normal retention yields higher adversarial reward and tail but lower nominal progress and Test14-random R score.

Limitations

• Limited by fidelity of the tokenized world model.
• No exhaustive scenario coverage.
• No formal closed-loop safety guarantees.

Experimental setup

Datasets: nuPlan InterPlan

Benchmarks: nuPlan Val14 nuPlan Test14-hard nuPlan Test14-random InterPlan InterPlan10 InterPlan-LongTail

Metrics: closed-loop score (NR) closed-loop score (R) tail-CVaR adversarial reward R-Score drop under AWM

Baselines: Plan-R1 PlanTF PLUTO Flow Planner Diffusion Planner PDM-Closed PlannerRFT Simultaneous alternating self-play

Builds on: Plan-R1 GRPO CVaR

Closest papers by meaning

• Preference-Guided Adversarial Policy Optimization for Long-Tail Robust Driving (NeurIPS 2026 · similarity 0.90)
• AdaWM: Adaptive World Model based Planning for Autonomous Driving (ICLR 2025 · similarity 0.87)
• Robust Multi-Agent Reinforcement Learning with Stochastic Adversary (ICML 2025 · similarity 0.86)
• MAPLE: Latent Multi-Agent Play for End-to-End Autonomous Driving (NeurIPS 2026 · similarity 0.86)
• Learning to Drive in New Cities Without Human Demonstrations (NeurIPS 2026 · similarity 0.85)
• UniMotion: A Unified Motion Framework for Simulation, Prediction and Planning (NeurIPS 2025 · similarity 0.85)