Research map / Video / Video generation
FlowMo: Variance-Based Flow Guidance for Coherent Motion in Video Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FlowMo, a training-free inference-time guidance that reduces the maximal patch-wise temporal variance of appearance-debiased latent predictions in early-mid denoising steps, improving motion coherence of pre-trained text-to-video DiT models.
Problem
Text-to-video diffusion models produce temporal artifacts (extra limbs, objects appearing/disappearing, abrupt motion discontinuities) and struggle with motion coherence; existing fixes require retraining or external motion signals.
Approach
Derive an appearance-debiased temporal representation from the model's own velocity predictions (l1 distance between consecutive frames), measure per-patch temporal variance, and perform gradient-based refinement of the input latent at early-to-mid denoising steps to reduce the maximal variance.
Key results
- On Wan2.1-1.3B, FlowMo raises VBench Final Score from 75.14% to 81.34% (+6.20%) and Motion Smoothness from 96.43% to 98.56%.
- On CogVideoX-5B, FlowMo raises VBench Final Score from 65.43% to 70.69% (+5.26%) and Motion Smoothness from 95.01% to 97.29%.
- In a human study on VideoJAM-bench, FlowMo was preferred for motion coherence in 44.3% of Wan2.1 comparisons vs 16.2% for baseline; adapted FreeInit reduced Dynamic Degree by 16.20%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply FlowMo guidance | Wan2.1-1.3B · VBench | Motion Smoothness: 96.43% -> 98.56% | Helped |
| Apply FlowMo guidance | Wan2.1-1.3B · VBench | Final Score: 75.14% -> 81.34% (+6.20%) | Helped |
| Apply FlowMo guidance | CogVideoX-5B · VBench | Motion Smoothness: 95.01% -> 97.29% | Helped |
| Apply FlowMo guidance | CogVideoX-5B · VBench | Final Score: 65.43% -> 70.69% (+5.26%) | Helped |
| Apply FlowMo guidance | Wan2.1-1.3B · VBench | Dynamic Degree: 83.21% -> 81.96% | Hurt |
| Apply FlowMo guidance | CogVideoX-5B · VBench | Dynamic Degree: 65.29% -> 63.92% | Hurt |
| Apply FlowMo guidance | Wan2.1-1.3B · VideoJAM-bench | Motion Coherence preference: 44.3% (vs. 16.2% for baseline) | Helped |
| Apply FlowMo guidance | Wan2.1-1.3B · VideoJAM-bench | Aesthetic Quality preference: 31.1% (vs. 14.0% for baseline) | Helped |
| Apply FlowMo guidance | CogVideoX-5B · VideoJAM-bench | Text-Video Alignment preference: 15.6% (vs. 8.7% for baseline) | Helped |
| Replace the maximum patch variance with the mean (ablation) | FlowMo on Wan2.1-1.3B · Wan2.1-1.3B generations | Motion coherence | Hurt |
| Remove the appearance-debiasing operator (ablation) | FlowMo on Wan2.1-1.3B · Wan2.1-1.3B generations | Motion coherence | Hurt |
| Apply FlowMo across all diffusion steps instead of steps 1-12 (ablation) | FlowMo on Wan2.1-1.3B · Wan2.1-1.3B generations | Visual artifacts | Hurt |
Surprising findings
- Adapted FreeInit lowers the VBench Final Score slightly below baseline (75.06% vs 75.14%) on Wan2.1 while cutting dynamic degree by 16.2%.
- Applying guidance across all diffusion steps introduces artifacts, so restricting optimization to early-to-mid steps is necessary.
Limitations
- Inference is about 2.39x slower due to gradient computation and propagation at inference time.
- Bounded by the pre-trained model's learned motion capabilities; cannot synthesize motion types the model has not learned.
- Dynamic degree drops slightly for both models (under 1.5%).
Experimental setup
Datasets: VideoJAM-bench
Benchmarks: VBench VideoJAM-bench
Metrics: Motion Smoothness Dynamic Degree Semantic Score Quality Score Final Score human preference rate
Baselines: Wan2.1-1.3B CogVideoX-5B FreeInit
Builds on: flow matching classifier-free guidance FreeInit VBench
Closest papers by meaning
- VideoJAM: Joint Appearance-Motion Representations for Enhanced Motion Generation in Video Models (ICML 2025 · similarity 0.89)
- Optical-Flow Guided Prompt Optimization for Coherent Video Generation (CVPR 2025 · similarity 0.89)
- Go-with-the-Flow: Motion-Controllable Video Diffusion Models Using Real-Time Warped Noise (CVPR 2025 · similarity 0.89)
- Emergent Temporal Correspondences from Video Diffusion Transformers (NeurIPS 2025 · similarity 0.88)
- LaMo: Self-Supervised Latent Motion Priors for Physical Realism in Video Generation (NeurIPS 2026 · similarity 0.88)
- FlowVid: Taming Imperfect Optical Flows for Consistent Video-to-Video Synthesis (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.