Research map / Video / Video generation
ConMo: Controllable Motion Disentanglement and Recomposition for Zero-Shot Motion Transfer
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ConMo, a zero-shot text-to-video motion transfer framework that disentangles subject-specific and background motion via masks and recomposes them with a soft guidance weight for multi-subject and shape-changing edits.
Problem
Zero-shot motion transfer from a reference video struggles with multi-subject videos (subject motions entangled) and with transferring motion to subjects of different shape or size, where holistic motion cues constrain the generated shape.
Approach
Compute local space-time feature differences inside subject masks (excluding overlaps with other subjects) and background masks during DDIM inversion, then recompose these motion cues as guidance during denoising, with a soft guidance weight that dilutes subject motion by blending in background (camera) motion.
Key results
- ConMo reaches 31.96 CLIP text alignment and 0.8931 motion fidelity, versus DMT at 31.46 and 0.8815 in Table 1.
- User study with 25 participants rates ConMo 4.40 motion preservation, 4.11 motion quality, 4.30 text alignment, the highest among compared methods.
- Ablation shows motion fidelity rises from 0.8675 (DMT) to 0.8931 with the full pipeline (Eq.3 plus soft guidance).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ConMo instead of DMT | Motion transfer · ConMo evaluation set | Motion Fidelity: 0.8815 -> 0.8931 | Helped |
| Use ConMo instead of VMC | Motion transfer · ConMo evaluation set | Text Alignment: 32.56 -> 31.96 | Hurt |
| Use ConMo instead of MotionClone | Motion transfer · ConMo evaluation set | Text Alignment: 31.00 -> 31.96 | Helped |
| Use ConMo instead of DMT | Motion transfer · ConMo evaluation set | Motion Quality: 3.70 -> 4.11 | Helped |
| Replace global guidance with coarse local motion extraction (Eq.1) (ablation) | DMT motion transfer · ConMo evaluation set | Motion Fidelity: 0.8675 -> 0.8813 | Helped |
| Replace global guidance with coarse local motion extraction (Eq.1) (ablation) | DMT motion transfer · ConMo evaluation set | Text Alignment: 31.46 -> 31.55 | Helped |
| Add soft guidance (ablation) | DMT + Eq.1 · ConMo evaluation set | Text Alignment: 31.55 -> 31.89 | Helped |
| Add soft guidance (ablation) | DMT + Eq.1 · ConMo evaluation set | Motion Fidelity: 0.8813 -> 0.8795 | Hurt |
| Use refined subject motion extraction with Eq.3 instead of coarse extraction with Eq.1 (ablation) | DMT + SG · ConMo evaluation set | Motion Fidelity: 0.8795 -> 0.8931 | Helped |
| Use refined subject motion extraction with Eq.3 instead of coarse extraction with Eq.1 (ablation) | DMT + SG · ConMo evaluation set | Text Alignment: 31.89 -> 31.96 | Helped |
Surprising findings
- Background motion alone reproduces camera movement, enabling camera motion simulation from the same disentangled cues.
Limitations
- Increasing background motion weight progressively loses original subject shape details.
- Evaluation relies on a small custom set of 26 videos and a 25-person user study.
Experimental setup
Datasets: DAVIS TGVE web-collected videos
Benchmarks: custom 26-video, 56 edited text-video pair set
Metrics: CLIP text alignment Motion Fidelity Score user study motion preservation/quality/text alignment (1-5)
Baselines: Control-A-Video VMC MotionClone DMT
Builds on: Diffusion-Motion-Transfer (DMT) SAM2 DDIM inversion VMC
Closest papers by meaning
- MotionClone: Training-Free Motion Cloning for Controllable Video Generation (ICLR 2025 · similarity 0.87)
- MotionBooth: Motion-Aware Customized Text-to-Video Generation (NeurIPS 2024 · similarity 0.87)
- DisMo: Disentangled Motion Representations for Open-World Motion Transfer (NeurIPS 2025 · similarity 0.87)
- Wan-Move: Motion-controllable Video Generation via Latent Trajectory Guidance (NeurIPS 2025 · similarity 0.87)
- Space-Time Diffusion Features for Zero-Shot Text-Driven Motion Transfer (CVPR 2024 · similarity 0.87)
- IM-Zero: Instance-level Motion Controllable Video Generation in a Zero-shot Manner (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.