Research map / Video / Video generation
MotionPro: A Precise Motion Controller for Image-to-Video Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MotionPro, an SVD-based image-to-video controller that conditions denoising on region-wise sparse trajectories and a motion mask via feature modulation, and introduces the MC-Bench benchmark.
Problem
Trajectory-based controllable image-to-video generation relies on large Gaussian-filtered trajectories, which yields coarse fine-grained motion control and cannot reliably disambiguate object motion from camera motion.
Approach
Replace Gaussian-extended trajectories with region-wise trajectories sampled from local k x k regions of tracked flow maps, add a thresholded motion mask to indicate object vs. camera motion, and inject both via adaptive feature modulation (scale/bias) into SVD's 3D-UNet, with LoRA on attention layers.
Key results
- On WebVid-10M, MotionPro reaches FVD 59.88 vs. 87.70 for MOFA-Video and 96.65 for DragNUWA.
- On MC-Bench object-level control, MotionPro gets MD-Img 10.48 and MD-Vid 8.59, reducing the best competitor DragAnything's distances by 1.82 and 2.78.
- On MC-Bench fine-grained control, MotionPro achieves MD-Vid 8.34 vs. 10.50 for MOFA-Video.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use region-wise trajectories and a motion mask for control | MotionPro · WebVid-10M | FVD: 87.70 -> 59.88 | Helped |
| Use region-wise trajectories and a motion mask for control | MotionPro · MC-Bench fine-grained | MD-Vid: 10.50 -> 8.34 | Helped |
| Use region-wise trajectories and a motion mask for control | MotionPro · MC-Bench object-level | MD-Vid: 11.37 -> 8.59 | Helped |
| Increase local region size k (ablation) | MotionPro · MC-Bench fine-grained and object-level | MD-Vid | Mixed |
| Set local region size to k=8 (ablation) | MotionPro · MC-Bench fine-grained and object-level | MD-Vid | Helped |
| Increase the minimal trajectory mask ratio rmin (ablation) | MotionPro · MC-Bench fine-grained and object-level | MD-Vid | Mixed |
| Use adaptive feature modulation instead of feature concatenation or summation (ablation) | MotionPro · MC-Bench | MD-Vid | Helped |
Surprising findings
- MOFA-Video obtains slightly higher Frame Consistency than MotionPro on MC-Bench fine-grained control (0.9972 vs. 0.9962), which the authors attribute to lower motion magnitude.
Limitations
- Relies on an optical tracking model (DOT) to derive training flow and masks, so training quality depends on tracker accuracy.
- Trained with a single minimal mask ratio range; large mask ratios make optimization harder with sparse trajectories.
Experimental setup
Datasets: WebVid-10M MC-Bench
Benchmarks: WebVid-10M MC-Bench
Metrics: FVD FID Frame Consistency MD-Img MD-Vid average flow magnitude
Baselines: DragNUWA MOFA-Video DragDiffusion DragAnything
Builds on: Stable Video Diffusion DragNUWA MOFA-Video Dense Optical Tracking LoRA
Closest papers by meaning
- Wan-Move: Motion-controllable Video Generation via Latent Trajectory Guidance (NeurIPS 2025 · similarity 0.92)
- Follow-Your-Click: Open-domain Regional Image Animation via Motion Prompts (AAAI 2025 · similarity 0.90)
- SG-I2V: Self-Guided Trajectory Control in Image-to-Video Generation (ICLR 2025 · similarity 0.89)
- Boximator: Generating Rich and Controllable Motions for Video Synthesis (ICML 2024 · similarity 0.89)
- Trajectory attention for fine-grained video motion control (ICLR 2025 · similarity 0.89)
- Image Conductor: Precision Control for Interactive Video Synthesis (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.