Research map / Video / Video generation
Image Conductor: Precision Control for Interactive Video Synthesis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Image Conductor enables precise, separate control of camera transitions and object movements in image-to-video generation by training camera and object LoRAs on a ControlNet with an orthogonal loss, plus a camera-free guidance at inference.
Problem
Generating videos from a single image with fine-grained, disentangled control over camera transitions and object motion, since real-world video data mixes both types of motion and existing trajectory-based methods confuse them.
Approach
Train separate camera and object LoRA weights on a motion ControlNet (object LoRA trained on top of frozen camera LoRA with stop-gradient and an orthogonal loss), and at inference extrapolate between model outputs with and without the object LoRA (camera-free guidance) to enhance object motion while suppressing camera transitions.
Key results
- Image Conductor achieves FID 209.74 and FVD 1116.17, versus 237.26/1283.85 for DragNUWA and 243.17/1287.15 for DragAnything.
- Image Conductor achieves CamMC 33.49 and ObjMC 42.38 (lower is better), outperforming DragNUWA (48.72/51.24) and DragAnything (66.54/60.97).
- In a 31-participant human study, Image Conductor is preferred overall in 61.7% of comparisons versus 31.8% for DragNUWA and 6.5% for DragAnything.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Image Conductor instead of DragNUWA | Image-to-video motion control · Camera-Only and Object-Only Motion Evaluation Datasets | FID: 237.26 -> 209.74 | Helped |
| Use Image Conductor instead of DragNUWA | Image-to-video motion control · Camera-Only and Object-Only Motion Evaluation Datasets | FVD: 1283.85 -> 1116.17 | Helped |
| Use Image Conductor instead of DragNUWA | Image-to-video motion control · Camera-Only and Object-Only Motion Evaluation Datasets | CamMC: 48.72 -> 33.49 | Helped |
| Use Image Conductor instead of DragNUWA | Image-to-video motion control · Camera-Only and Object-Only Motion Evaluation Datasets | ObjMC: 51.24 -> 42.38 | Helped |
| Use Image Conductor instead of DragNUWA | Image-to-video motion control · Camera-Only and Object-Only Motion Evaluation Datasets | Overall preference: 31.8% -> 61.7% | Helped |
| Use Image Conductor instead of DragNUWA | Image-to-video motion control · Camera-Only and Object-Only Motion Evaluation Datasets | Quality preference: 37.1% -> 54.8% | Helped |
| Use Image Conductor instead of DragNUWA | Image-to-video motion control · Camera-Only and Object-Only Motion Evaluation Datasets | Motion preference: 27.7% -> 66.0% | Helped |
| Use distinct camera and object LoRA weights instead of a shared motion control (ablation) | Image Conductor · Camera and object motion control | Motion separation | Helped |
| Add camera-free guidance during inference (ablation) | Image Conductor · Challenging multi-trajectory object-motion examples | Object motion control | Helped |
| Set camera-free guidance λ_trajs to 1 instead of using camera-free guidance (ablation) | Image Conductor · Challenging multi-trajectory object-motion examples | Camera transition suppression | Hurt |
| Increase camera-free guidance λ_trajs to 1.1 (ablation) | Image Conductor · Challenging multi-trajectory object-motion examples | Object motion control | Helped |
Limitations
- Generated content quality is constrained by the base AnimateDiff model, with color and detail inconsistencies in later frames.
- Conflicting text and image prompts can compromise output quality.
- Camera-free guidance still shows artifacts at modest guidance values, and multi-trajectory camera transitions are hard to separate with LoRA alone.
Experimental setup
Datasets: WebVid RealEstate10K
Benchmarks: WebVid Camera-only motion evaluation set Object-only motion evaluation set
Metrics: FID FVD CamMC ObjMC human preference (overall, quality, motion)
Baselines: DragNUWA DragAnything MotionCtrl
Builds on: AnimateDiff SparseCtrl ControlNet LoRA classifier-free guidance
Closest papers by meaning
- MotionPro: A Precise Motion Controller for Image-to-Video Generation (CVPR 2025 · similarity 0.88)
- Wan-Move: Motion-controllable Video Generation via Latent Trajectory Guidance (NeurIPS 2025 · similarity 0.88)
- Boximator: Generating Rich and Controllable Motions for Video Synthesis (ICML 2024 · similarity 0.88)
- TokenMotion: Decoupled Motion Control via Token Disentanglement for Human-centric Video Generation (CVPR 2025 · similarity 0.87)
- SG-I2V: Self-Guided Trajectory Control in Image-to-Video Generation (ICLR 2025 · similarity 0.87)
- I2VControl-Camera: Precise Video Camera Control with Adjustable Motion Strength (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.