Research map / Multi-agent systems and games / Human-AI and goal-conditioned
UniMotion: A Unified Motion Framework for Simulation, Prediction and Planning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes UniMotion, a decoder-only Transformer that jointly handles motion simulation, trajectory prediction and ego planning on Waymo Open Motion Dataset via task-specific attention masks, joint NTP+LFR training, and per-task fine-tuning.
Problem
Motion simulation, trajectory prediction, and ego planning in autonomous driving are typically addressed by specialized models, hindering cross-task generalization and missing potential mutual benefits.
Approach
Abstract motion tasks into diverse motion generation and long-range trajectory forecasting, and unify them in one GPT-style decoder-only Transformer with task-specific attention masks, next-token prediction plus long-range future regression losses, and parameter-free task-specific fine-tuning (GRPO-style RL for simulation, multi-modal intention-anchor decoder for prediction, pred2gen two-stage fin
Key results
- On the 2025 Waymo Open Sim Agents Challenge, UniMotion reaches Realism Meta Metric 0.7851 and minADE 1.3036, the best realism score among compared methods (CATK 0.7846, UniMM 0.7829).
- On the WOMD Prediction leaderboard, UniMotion achieves minADE 0.5718 and minFDE 1.1643, with mAP 0.4534 (best) though Soft mAP (0.4642) trails RMP-YOLO.
- On Waymo planning, UniMotion reports planning error @1s 0.083, @3s 0.591, @5s 2.246 and collision 1.565, outperforming DIPP (0.227, 1.187, 3.335 errors).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| jointly train with NTP and LFR rather than LFR alone (ablation) | UniMotion · WOMD validation split | Prediction minADE: 0.6668 -> 0.6508 | Helped |
| jointly train with NTP and LFR rather than NTP alone (ablation) | UniMotion · WOMD validation split | Simulation Kinematic: 0.4884 -> 0.4892 | Helped |
| add simulation fine-tuning (ablation) | UniMotion · WOMD validation split | Kinematic: 0.4892 -> 0.4939 | Helped |
| add prediction fine-tuning with intention anchors (ablation) | UniMotion · WOMD validation split | mAP: 0.3147 -> 0.3856 | Helped |
| add Pred2Gen planning fine-tuning (ablation) | UniMotion · WOMD validation split | Planning error @1s: 0.1326 -> 0.0834 | Helped |
| add closed-loop consistency constraint (ablation) | UniMotion simulation fine-tuning · WOMD validation split | Interactive: 0.7968 -> 0.8019 | Helped |
| add RL supervision without consistency constraint (ablation) | UniMotion simulation fine-tuning · WOMD validation split | Map: 0.9144 -> 0.9120 | Hurt |
| combine consistency constraint with RL supervision (ablation) | UniMotion simulation fine-tuning · WOMD validation split | Interactive: 0.7968 -> 0.8032 | Helped |
| remove intention anchors from prediction fine-tuning (ablation) | UniMotion prediction fine-tuning · WOMD validation split | minADE: 0.6413 -> 0.6174 | Helped |
| remove intention anchors from prediction fine-tuning (ablation) | UniMotion prediction fine-tuning · WOMD validation split | mAP: 0.3856 -> 0.3389 | Hurt |
| fine-tune UniMotion for simulation | UniMotion · 2025 Waymo Open Sim Agents Challenge | Realism Meta Metric | Helped |
| fine-tune UniMotion for prediction | UniMotion · WOMD Prediction Leaderboard | Soft mAP | Hurt |
Surprising findings
- RL-only fine-tuning with DAPO-style rewards fails to give satisfactory simulation results, while closed-loop consistency supervision yields most of the gain.
- End-to-end prediction fine-tuning without intention anchors reduces minADE/MR further but degrades mAP, so anchors are retained despite worse regression error.
Limitations
- Coupled spatial-temporal interaction and relative positional embeddings rely on detailed positional information, limiting use of LLM-related techniques.
- Trained only on logged trajectories, leading to suboptimal reaction to red lights in planning.
- Failure cases include collisions in crowded parking lots, stationary agents at green lights, and difficulty with sudden turns and U-turns.
Experimental setup
Datasets: Waymo Open Motion Dataset
Benchmarks: Waymo Open Sim Agents Challenge 2025 WOMD Prediction Leaderboard Waymo Planning benchmark WOMD validation split
Metrics: Realism Meta Metric Kinematic metrics Interactive metrics Map-based metrics minADE minFDE Miss Rate mAP Soft mAP Collision
Baselines: UniMM CATK UniTAM UniTFormer LLM2AD HDGT MTR MTR++ MGTR EDA ControlMTR RMP-YOLO
Builds on: GRPO DAPO Sim Agents benchmark MTR Closest Among Top-K (CATK)
Closest papers by meaning
- AlignDrive: Aligned Lateral-Longitudinal Planning for End-to-End Autonomous Driving (NeurIPS 2026 · similarity 0.89)
- DeMo: Decoupling Motion Forecasting into Directional Intentions and Dynamic States (NeurIPS 2024 · similarity 0.88)
- DriveGPT: Scaling Autoregressive Behavior Models for Driving (ICML 2025 · similarity 0.88)
- BehaviorGPT: Smart Agent Simulation for Autonomous Driving with Next-Patch Prediction (NeurIPS 2024 · similarity 0.88)
- Flow Matching-Based Autonomous Driving Planning with Advanced Interactive Behavior Modeling (NeurIPS 2025 · similarity 0.87)
- Trajeglish: Traffic Modeling as Next-Token Prediction (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.