Research map / Video / Video generation
LaMo: Self-Supervised Latent Motion Priors for Physical Realism in Video Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LaMo, a self-supervised latent motion prior over frame-to-frame latent changes that yields a training-time Motion Drift Loss and a sampling-time Motion Prior Guidance to improve physical fidelity of CogVideoX video diffusion models.
Problem
Video diffusion models produce visually plausible clips that often violate physical and motion consistency, and existing fixes rely on external simulators, teacher models, or curated physics data.
Approach
Extract a latent motion prior from frame-to-frame latent differences in ordinary unlabeled training videos, exposing a channel-mean macro drift as a training loss on predicted x0 and a learned small CNN micro motion field as CFG-style guidance on the noise prediction during sampling.
Key results
- On VideoPhy, LaMo-2B improves CogVideoX-2B overall SA/PC from 60.5/25.6 to 67.2/31.4.
- On VideoPhy2, LaMo-2B reaches 25.4 SA / 75.4 PC versus 21.0/68.0 for CogVideoX-2B and 21.0/72.5 for VideoREPA-2B.
- On VBench, LaMo raises CogVideoX total score from 78.2 to 79.6, with Spatial Relationship rising from 52.3 to 62.2.
Surprising findings
- Applying guidance directly to the latent z_t drops performance below the unaugmented backbone (62.8 SA / 31.6 PC vs 64.8/35.2 baseline).
- Dense per-pixel motion supervision degrades results relative to the channel-mean macro drift target.
Limitations
- Not a constraint-satisfying simulator; gains bounded by video data coverage.
- VideoPhy-style model-judged metrics do not isolate contact, deformation, fluids, conservation, or long-horizon stability.
Experimental setup
Datasets: OpenVid VideoPhy VideoPhy2 VBench
Benchmarks: VideoPhy VideoPhy2 VBench
Metrics: Semantic Adherence Physical Commonsense VBench quality score VBench semantic score VBench total score
Baselines: CogVideoX-2B CogVideoX-5B DreamWorld MoAlign VideoREPA PhyT2V WISA PHANTOM
Builds on: CogVideoX VideoREPA classifier-free guidance LoRA
Closest papers by meaning
- FlowMo: Variance-Based Flow Guidance for Coherent Motion in Video Generation (NeurIPS 2025 · similarity 0.88)
- Physics-Informed Video Generation via Mixture-of-Experts Latent Alignment (NeurIPS 2026 · similarity 0.88)
- PhysCtrl: Generative Physics for Controllable and Physics-Grounded Video Generation (NeurIPS 2025 · similarity 0.88)
- MotionCraft: Physics-Based Zero-Shot Video Generation (NeurIPS 2024 · similarity 0.87)
- TTVidT: Decoupling the Temporal Axis for Efficient Motion-Centric Video Pretraining (NeurIPS 2026 · similarity 0.87)
- Efficient Video Diffusion Models via Content-Frame Motion-Latent Decomposition (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.