Research map / Video / Video generation
EA-WM: Event-Aware Generative World Model with Structured Kinematic-to-Visual Action Fields
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes EA-WM, a Wan2.2-based robotic video world model that renders robot actions and kinematics as camera-aligned visual fields (KVAFs) and fuses them with video features via event-gated bidirectional attention supervised by frame-difference latents (EDLS), reporting state-of-the-art P3CScore on WorldArena.
Problem
Video world models conditioned on low-dimensional robot actions often fail to preserve precise robot geometry and fine-grained robot-object interaction dynamics in generated rollouts.
Approach
Lift joint/end-effector actions and kinematic states into the target camera view as rendered visual action fields (KVAFs) processed by a parallel DiT branch, and use a per-token event gate, supervised by VAE-encoded frame-difference latents, to modulate bidirectional cross-attention between the video and KVAF streams.
Key results
- EA-WM achieves P3CScore 76.60, outperforming the strongest baseline CogVideoX (71.08) by 5.52 points on WorldArena selected metrics.
- Ablation: removing KVAFs drops P3CScore to 70.97 and removing event-aware fusion drops it to 74.80 versus the full model's 76.60.
- The heuristic KVAF-to-action recovery pipeline has a detection rate of about 0.45, and numerical action prediction performs better on recovery errors.
Surprising findings
- Heuristic recovery of numerical actions from generated KVAF videos is weaker than direct numerical action prediction, with only about 0.45 detection rate.
- The variant without event-aware fusion slightly outperforms the full EA-WM on Semantic Alignment (0.896 vs 0.895).
Limitations
- KVAFs require robot kinematics, camera calibration, and synchronized action-state logs, which may not hold in real-world deployment.
- KVAFs encode mainly robot-side geometry; object-state changes are captured only indirectly through event-aware fusion and EDLS.
- Evaluation is limited to three WorldArena dimensions (six metrics) in the main paper and to RoboTwin data.
Experimental setup
Datasets: RoboTwin 2.0 WorldArena
Benchmarks: WorldArena
Metrics: P3CScore Interaction Quality Trajectory Accuracy Depth Accuracy Perspectivity Instruction Following Semantic Alignment Avg16
Baselines: CogVideoX WoW Wan 2.2 TesserAct GigaWorld-0 Vidar Cosmos-Predict 2.5 Genie Envisioner CtrlWorld IRASim RoboMaster
Builds on: Wan2.2 flow matching LoRA DreamZero
Closest papers by meaning
- Action Images: End-to-End Policy Learning via Multiview Video Generation (NeurIPS 2026 · similarity 0.91)
- World Motion Models: Flexible Sequence Modeling of SE(3) Trajectories (NeurIPS 2026 · similarity 0.89)
- Learning Visual Feature-Based World Models via Residual Latent Action (NeurIPS 2026 · similarity 0.89)
- DiT4DiT: Jointly Modeling Video Dynamics and Actions for Generalizable Robot Control (NeurIPS 2026 · similarity 0.89)
- SyncWorld: Visual Calibration Enables World Models as Zero-Shot Simulators (NeurIPS 2026 · similarity 0.89)
- SAMPO: Scale-wise Autoregression with Motion Prompt for Generative World Models (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.