Research map / Video / Video generation
Wan-Move: Motion-controllable Video Generation via Latent Trajectory Guidance
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Wan-Move adds motion control to the Wan-I2V-14B image-to-video model by replicating first-frame latent features along latent point trajectories (no extra modules), and introduces the MoveBench benchmark.
Problem
Motion-controllable video generation suffers from coarse control granularity and from auxiliary motion encoders/ControlNets that degrade motion signals and hinder scalable fine-tuning of video backbones.
Approach
Transfer pixel-space point trajectories into latent coordinates and propagate the first frame's VAE latent features along each trajectory, so the edited image condition itself carries motion; this is concatenated to the input of an off-the-shelf I2V DiT with no architecture change.
Key results
- On MoveBench single-object, Wan-Move reaches FID 12.2, FVD 83.5, EPE 2.6 versus best baseline Tora/LeviTor EPE ~3.3-3.4 and MagicMotion FVD 96.7.
- In a 2AFC human study, Wan-Move beats Tora with win rates above 93.8% on all three criteria and is competitive with Kling 1.5 Pro, winning 53.4% on motion quality.
- Latent feature replication outperforms pixel replication (EPE 3.7) and random track embedding (EPE 2.7), and concatenation matches ControlNet quality (FVD 83.5 vs 84.6) at far lower latency.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace pixel replication with latent feature replication for motion guidance (ablation) | Wan-Move · MoveBench | EPE: 3.7 -> 2.6 | Helped |
| Replace random track embeddings with latent feature replication (ablation) | Wan-Move · MoveBench | EPE: 2.7 -> 2.6 | Helped |
| Use ControlNet fusion instead of concatenation (ablation) | Wan-Move · MoveBench | FID: 12.2 -> 12.4 | No clear effect |
| Increase the maximum training trajectory count from 10 to 200 (ablation) | Wan-Move · MoveBench | EPE: 3.3 -> 2.6 | Helped |
| Increase maximum training trajectories from 200 to 500 or 1024 (ablation) | Wan-Move · MoveBench | EPE: 2.6 -> 3.0; 3.9 | Hurt |
| Add point trajectories at inference, from none to one track (ablation) | Wan-Move · MoveBench | EPE: 12.4 -> 2.6 | Helped |
| Increase inference trajectories from 1 to 1024 (ablation) | Wan-Move · MoveBench | EPE: 2.6 -> 1.1 | Helped |
| Randomly select one overlapping trajectory feature rather than average features (ablation) | Wan-Move · MoveBench | EPE: 2.7 -> 2.6 | Helped |
| Use direct sampling rather than dense-to-sampling training (ablation) | Wan-Move · MoveBench | FVD: 84.2 -> 83.5 | Helped |
| Train Wan-Move-Cog-23K rather than MagicMotion with the same backbone and data scale | CogVideoX-5B · MoveBench | EPE: 3.2 -> 2.8 | Helped |
| Train Wan-Move-Cog-630K rather than Tora with the same backbone and data scale | CogVideoX-5B · MoveBench | EPE: 3.3 -> 2.8 | Helped |
Surprising findings
- Naive I2V inference with no point tracks yields PSNR and SSIM comparable to motion-controlled generation, with the model retaining base I2V quality.
- Simple channel concatenation of the edited condition matches ControlNet performance while avoiding ControlNet's large latency overhead.
Limitations
- Relies on point trajectories that can be unreliable under prolonged occlusion, leading to loss of control.
- Performance degrades in visually crowded scenes with many interacting objects and on physically implausible trajectories.
- Erroneous CoTracker tracks can compound failure modes.
Experimental setup
Datasets: Panda70M Pixabay Pexels YouTube DAVIS MoveBench
Benchmarks: MoveBench DAVIS
Metrics: FID FVD PSNR SSIM EPE
Baselines: ImageConductor LeviTor Tora MagicMotion Kling 1.5 Pro
Builds on: Wan2.1 CoTracker Motion Prompting flow matching SAM
Closest papers by meaning
- MotionPro: A Precise Motion Controller for Image-to-Video Generation (CVPR 2025 · similarity 0.92)
- SG-I2V: Self-Guided Trajectory Control in Image-to-Video Generation (ICLR 2025 · similarity 0.90)
- CAGE: Unsupervised Visual Composition and Animation for Controllable Video Generation (AAAI 2025 · similarity 0.90)
- IM-Zero: Instance-level Motion Controllable Video Generation in a Zero-shot Manner (CVPR 2025 · similarity 0.90)
- Boximator: Generating Rich and Controllable Motions for Video Synthesis (ICML 2024 · similarity 0.90)
- Training-free Camera Control for Video Generation (ICLR 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.