Research map / Video / Video generation
ReVideo: Remake a Video with Motion and Content Control
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ReVideo, an SVD-based method for local video editing that jointly controls edited first-frame content and trajectory-based motion in a masked region, using a three-stage coarse-to-fine training strategy and a spatiotemporal adaptive fusion module.
Problem
Accurate localized video editing that modifies both visual content and motion trajectories in specific areas while keeping unedited content unchanged; existing methods handle mostly content, not motion, and suffer from training imbalance and condition coupling between content and motion controls.
Approach
Edit content by modifying the first frame and specify motion with trajectory lines; train control modules in three stages (motion prior, decoupled training with two different source videos in edited/unedited regions, deblocking) and fuse content and motion conditions with a learned weight map conditioned on mask and timestep (SAFM).
Key results
- Human evaluation: ReVideo preferred overall in 59.1% and for editing target in 67.0% of samples, versus Pika at 27.9% and 23.9%.
- ReVideo achieves the highest text alignment (0.2304 CLIP) among compared methods, while Pika has slightly higher PSNR (33.07 vs 32.85) and temporal consistency (0.9956 vs 0.9864).
- The method extends to multi-area editing without specific training and to 90-frame videos via a sliding window.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ReVideo instead of Pika | Local video editing · 16-video test set | Text Alignment: 0.2184 -> 0.2304 | Helped |
| Use ReVideo instead of Pika | Local video editing · 16-video test set | Overall: $27.9\%$ -> $\mathbf{59.1\%}$ | Helped |
| Use ReVideo instead of Pika | Local video editing · 16-video test set | Editing Target: $23.9\%$ -> $\mathbf{67.0\%}$ | Helped |
| Use ReVideo instead of Pika | Local video editing · 16-video test set | PSNR: 33.07 -> 32.85 | Hurt |
| Use ReVideo instead of Pika | Local video editing · 16-video test set | Consistency: 0.9956 -> 0.9864 | Hurt |
| Replace SAFM with direct summing of motion and content control (ablation) | ReVideo · Complex motion trajectories | Trajectory control accuracy | Hurt |
| Remove the time condition from SAFM (ablation) | ReVideo · Editing-area boundaries | Boundary artifact quality | Hurt |
| Fine-tune the entire control module during deblocking training (ablation) | ReVideo · Local motion control | Motion control capability | Hurt |
| Fine-tune spatial layers during deblocking training (ablation) | ReVideo · Local motion control | Motion control capability | Hurt |
| Fine-tune key and value embeddings in temporal layers of the control module and base model (ablation) | ReVideo · Local motion control | Motion control capability | No clear effect |
| Fine-tune the base model's temporal key and value embeddings during stage 3 (ablation) | ReVideo · Complex and highly coupled editing scenes | Editing result harmony | Helped |
Surprising findings
- Static motion of edited content tends to score higher on CLIP-based temporal consistency, so consistency metrics can favor methods lacking motion control.
- Tuning the whole control module in the deblocking stage degrades motion control; tuning only temporal key/value embeddings preserves it.
Limitations
- Regeneration quality is bounded by the SVD base model; artifacts appear where SVD's generation prior is weak.
- Errors accumulate in long videos generated via sliding-window extension.
- Small evaluation set (16 videos) and human study with 20 volunteers.
Experimental setup
Datasets: WebVid
Benchmarks: custom 16-video test set (720x1280)
Metrics: PSNR CLIP text alignment CLIP temporal consistency human preference (overall quality, editing target)
Baselines: InsV2V AnyV2V Pika
Builds on: Stable Video Diffusion ControlNet EDM preconditioning CoTracker
Closest papers by meaning
- VideoDirector: Precise Video Editing via Text-to-Video Models (CVPR 2025 · similarity 0.88)
- MotionEditor: Editing Video Motion via Content-Aware Diffusion (CVPR 2024 · similarity 0.88)
- Re-Attentional Controllable Video Diffusion Editing (AAAI 2025 · similarity 0.86)
- Dynamic Motion Blending for Versatile Motion Editing (CVPR 2025 · similarity 0.86)
- Ctrl-Adapter: An Efficient and Versatile Framework for Adapting Diverse Controls to Any Diffusion Model (ICLR 2025 · similarity 0.86)
- Transforming Image Editors into Video Editors (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.