Research map / Video / Video generation
Transforming Image Editors into Video Editors
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Transforms strong image editors into video editors by editing sparse keyframes with an image editor and propagating edits via a motion-guided Wan2.2 image-to-video model with a lightweight motion encoder (AVE).
Problem
End-to-end instruction-guided video editing is expensive to train and data-hungry, and per-frame editing causes flicker and temporal inconsistency; the paper asks how to transfer mature image editing capability to video cheaply.
Approach
Decompose video editing into composed editing of sparse keyframes by a strong image editor, then interpolate the full video with a frozen image-to-video diffusion model whose anchor latents are clamped during denoising, adding only a lightweight motion encoder with a ControlNet-style branch.
Key results
- On IVEBench short subset, AVE achieves total score 70.58 vs. previous best 66.75 (Ditto).
- On IVEBench long subset, AVE reaches total 68.75 and motion 88.95, best among compared methods.
- Stage-1 editor ablation: Nano Banana 2 gives 68.75 total vs. 58.54 for untuned BAGEL on IVEBench-long, showing final quality tracks image editor quality.
Surprising findings
- Final video editing quality is strongly correlated with the quality of the Stage-1 image editor, with an untuned BAGEL causing a large drop (68.75 to 58.54 total).
- Keyframe selection rule matters little: uniform and scene-aware sampling differ only slightly across budgets.
Limitations
- Relies on a private video editing dataset for training the Stage-1 BAGEL editor and the Stage-2 motion branch, which is not released.
- Best results depend on closed-source image editors (Nano Banana 2, GPT-5.1, Seedream 4.0) that are not reproducible open-weight models.
- Long-video handling and keyframe budget are tied to the Wan2.2 VAE's 4x temporal downsampling; too many keyframes hurt performance.
Experimental setup
Datasets: private video editing dataset IVEBench VIE-Bench
Benchmarks: IVEBench VIE-Bench
Metrics: total score video quality instruction compliance fidelity motion smoothness semantic consistency content fidelity instruction following preservation quality
Baselines: Ditto InsV2V Lucy-Edit-Dev VACE ICVE Omni-Video AnyV2V StableV2V Kling1.6 Kling-Omni Runway Pika
Builds on: Wan2.2-I2V BAGEL ControlNet
Closest papers by meaning
- VINCIE-NExT: Unlocking Video Editing from Images via In-Context Modeling (NeurIPS 2026 · similarity 0.90)
- Fairy: Fast Parallelized Instruction-Guided Video-to-Video Synthesis (CVPR 2024 · similarity 0.89)
- Señorita-2M: A High-Quality Instruction-based Dataset for General Video Editing by Video Specialists (NeurIPS 2025 · similarity 0.88)
- Pathways on the Image Manifold: Image Editing via Video Generation (CVPR 2025 · similarity 0.88)
- MotionEditor: Editing Video Motion via Content-Aware Diffusion (CVPR 2024 · similarity 0.88)
- VideoDirector: Precise Video Editing via Text-to-Video Models (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.