Research map / 3D vision / 3D generation

World Motion Models: Flexible Sequence Modeling of SE(3) Trajectories

NeurIPS 2026 · 3D vision › 3D generation · approach: Scene-aware human motion synthesis · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes World Motion Models, flow-matching transformers with per-token noise levels over sparse SE(3) pose trajectories, using context tokens and any-marginal masking to serve as policies, world models, and retargeting/HOI generators across six applications.

Problem

Learning a unified generative prior over the dynamic 3D world that supports arbitrary known/unknown conditioning (prediction, policy, planning, retargeting) across heterogeneous entities such as robots, humans, objects, and scene points.

Approach

Represent every moving entity as sparse SE(3) pose tokens (9-D three-point parameterization) and train a flow-matching transformer with independent per-token noise levels plus a decoupled context-token timeline, so any mask of observed vs. target tokens yields a different conditional sampler from one network.

Key results

• On TraceGen EpicKitchen, WMM achieves MAE 2.167, MSE 0.329, endpoint 0.580 (vs. plain DiT MAE 32.05), with the paper claiming best performance in every column on both Droid and EpicKitchen.
• On Language Table B2B, the full WMM reaches 0.860 success rate, versus 0.800 without AdaLN and 0.000 without per-token noise.
• Humanoid retargeting at P=128 SR-seq 0.869 with gerr 0.594; the paper reports comparable or better HOI synthesis on OMOMO than CHOIS and a faster-than-GMR-quality tradeoff versus PHUMA.

Surprising findings

• The P=128 TraceGen variant, trained on only 128 of 400 trajectories per step, outperforms the P=400 variant on all metrics when inferring all 400 trajectories.
• Removing per-token noise yields zero successful Language Table rollouts, indicating synchronized noise with guidance fails in policy mode.

Limitations

• One checkpoint trained per benchmark; no joint cross-domain training.
• Requires geometry (depth, estimated states) as input rather than raw RGB video.
• Does not generalize to novel morphologies unseen during training.

Experimental setup

Datasets: Language Table TraceGen (Droid, EpicKitchen) OMOMO AMASS Embody3D TACO PyRoKi

Benchmarks: Language Table TraceGen OMOMO humanoid retargeting (G1, TWIST sim) TACO

Metrics: success rate MSE MAE endpoint MSE MPJPE tracking error

Baselines: ACT LAPA LCB TraceGen CHOIS GMR PHUMA

Builds on: Diffusion Forcing flow matching ACT AdaLN PointNet

Closest papers by meaning

• EA-WM: Event-Aware Generative World Model with Structured Kinematic-to-Visual Action Fields (NeurIPS 2026 · similarity 0.89)
• SAMPO: Scale-wise Autoregression with Motion Prompt for Generative World Models (NeurIPS 2025 · similarity 0.88)
• EnerVerse: Envisioning Embodied Future Space for Robotics Manipulation (NeurIPS 2025 · similarity 0.87)
• MolmoMotion: Forecasting Point Trajectories in 3D with Language Instruction (NeurIPS 2026 · similarity 0.87)
• PointZero: 3D Point Track Completion for Learning Transferable 3D Dynamics (NeurIPS 2026 · similarity 0.87)
• Learning Visual Feature-Based World Models via Residual Latent Action (NeurIPS 2026 · similarity 0.87)