Research map / Video / Action recognition
World Models as Group Actions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Formalizes action-conditioned video world modeling as a group action (SE(2) for navigation), and adds latent-space identity/inverse/composition regularization with synthesized action segments plus GAC and GAR metrics, improving NWM and DIAMOND on RECON.
Problem
Video world models achieve visual realism but their dynamics may not be faithfully governed by actions; existing evaluations ignore whether action compositional structure (identity, inverse, composition) is respected.
Approach
Treat action-conditioned world modeling as a group action of the action group (SE(2) for navigation) on the state space, and enforce identity, inverse and compatibility constraints as latent-space regularizers using synthesized zero-action, forward-inverse and Dirichlet-redistributed action segments, with recovered-state metrics GAC and GAR.
Key results
- On RECON, NWM+GA lowers aggregate GAC error from 1.73 to 1.50 versus NWM.
- GAR non-aligned error at 64 frames drops from 1.52 to 1.28 for NWM with GA regularization.
- DIAMOND+GA reduces GAC error from 1.99 to 1.76, showing gains beyond a single backbone.
Surprising findings
- Native NWM violates identity, inverse, and compatibility conditions despite visually plausible rollouts.
Limitations
- Composition consistency is only a local approximation since SE(2) composition is non-commutative and actions are additive increments.
- Recovered-state metrics depend on the DROID-SLAM pose estimator and stochastic generation.
- Latent losses are a surrogate for state-space regularization rather than an exact replacement.
Experimental setup
Datasets: RECON SCAND HuRoN
Benchmarks: RECON SCAND HuRoN
Metrics: Group-Action Consistency (GAC) Group-Action Robustness (GAR) identity error inverse error composition error
Baselines: NWM DIAMOND
Builds on: Navigation World Models DIAMOND DROID-SLAM
Closest papers by meaning
- Grounding Video Models to Actions through Goal Conditioned Exploration (ICLR 2025 · similarity 0.88)
- Trimming the Long-Tail of Visual World Modeling Evaluation (NeurIPS 2026 · similarity 0.88)
- Learning 3D Persistent Embodied World Models (NeurIPS 2025 · similarity 0.88)
- SyncWorld: Visual Calibration Enables World Models as Zero-Shot Simulators (NeurIPS 2026 · similarity 0.87)
- COMBO: Compositional World Models for Embodied Multi-Agent Cooperation (ICLR 2025 · similarity 0.87)
- Learning Visual Feature-Based World Models via Residual Latent Action (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.