Research map / Video / Video generation
MotionEditor: Editing Video Motion via Content-Aware Diffusion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MotionEditor, a one-shot diffusion-based video motion editing method that adds a content-aware motion adapter to ControlNet and uses training-free decoupled foreground/background attention injection to transfer reference motion while preserving source appearance and background.
Problem
Editing the motion of a protagonist in a video to match a reference video while preserving the original appearance and dynamic background, which existing diffusion video editors and pose-guided generators fail at due to conflicts between inverted source noise and pose control signals.
Approach
A content-aware motion adapter appended to ControlNet that cross-attends pose features to source frame latents so control signals are conditioned on source content, combined with a training-free two-branch attention injection that decouples keys/values into foreground and background using a segmentation mask, plus a skeleton alignment step for size and position.
Key results
- MotionEditor achieves CLIP 28.86 vs 28.07 for the next best (FateZero) on 20 in-the-wild cases, with LPIPS-S 0.273, LPIPS-N 0.124, LPIPS-T 0.082 (all best).
- User study: participants preferred MotionEditor over FateZero for motion alignment 78.9% of the time (i.e., the competitor's preference is below 50%, so MotionEditor wins), and over Follow-Your-Pose for appearance alignment at 85.0%.
- Each video takes about 10 minutes to edit on a single NVIDIA A100 GPU due to DDIM inversion and null-text optimization.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MotionEditor instead of FateZero | video motion editing · 20 in-the-wild cases | CLIP: 28.07 -> 28.86 | Helped |
| Use MotionEditor instead of Follow-Your-Pose | video motion editing · 20 in-the-wild cases | LPIPS-S: 0.337 -> 0.273 | Helped |
| Use MotionEditor instead of MotionEditor’s strongest baseline | video motion editing · 20 in-the-wild cases | LPIPS-N: 0.144 -> 0.124 | Helped |
| Use MotionEditor instead of FateZero | video motion editing · 20 in-the-wild cases | LPIPS-T: 0.124 -> 0.082 | Helped |
| Replace Consistent-Sparse Attention with Sparse Attention (ablation) | MotionEditor · motion editing videos | motion reliability | Hurt |
| Remove cross-attention from the motion adapter (ablation) | MotionEditor · motion editing videos | motion editing ability | Hurt |
| Remove the motion adapter (ablation) | MotionEditor · motion editing videos | motion editing ability | Hurt |
| Remove high-fidelity attention injection (ablation) | MotionEditor · motion editing videos | background preservation | Hurt |
| Remove skeleton alignment (ablation) | MotionEditor · motion editing videos | appearance preservation | Hurt |
Surprising findings
- Ablations show ControlNet alone gives weak motion constraints and fails to edit motion, indicating that conditioning on source content is needed for the control signal.
Limitations
- Temporal inconsistencies sometimes occur in foreground subjects.
- One-shot learning on a single video with no large-scale video pretraining.
- Slow inference (about 10 minutes per video) due to inversion and null-text optimization.
Experimental setup
Datasets: YouTube videos TaichiHD
Benchmarks: 20 in-the-wild video cases (YouTube and TaichiHD)
Metrics: CLIP score LPIPS-S LPIPS-N LPIPS-T user preference (motion, appearance, textual alignment)
Baselines: LWG MRAA Tune-A-Video Follow-Your-Pose ControlVideo MasaCtrl FateZero
Builds on: Tune-a-Video ControlNet Stable Diffusion MasaCtrl FateZero
Closest papers by meaning
- FADE: Frequency-Aware Diffusion Model Factorization for Video Editing (CVPR 2025 · similarity 0.89)
- SALAD: Skeleton-aware Latent Diffusion for Text-driven Motion Generation and Editing (CVPR 2025 · similarity 0.88)
- Motion Guidance: Diffusion-Based Image Editing with Differentiable Motion Estimators (ICLR 2024 · similarity 0.88)
- Transforming Image Editors into Video Editors (NeurIPS 2026 · similarity 0.88)
- Ctrl-Adapter: An Efficient and Versatile Framework for Adapting Diverse Controls to Any Diffusion Model (ICLR 2025 · similarity 0.88)
- Dynamic Motion Blending for Versatile Motion Editing (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.