Research map / Video / Video generation
CCEdit: Creative and Controllable Video Editing via Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
CCEdit is a diffusion-based video editing framework with a trident network (main T2V branch plus ControlNet structure branch and an edited-key-frame appearance branch) and introduces the BalanceCC benchmark.
Problem
Generative video editing struggles to balance precise controllability (structure preservation, fine-grained appearance control) with creative freedom across diverse edits such as style transfer, foreground and background modification.
Approach
A trident network decouples structure and appearance control: a frozen ControlNet branch injects per-frame structure maps, an appearance branch injects an edited key frame, and both feed into a T2I-derived main branch via learnable temporal layers; an anchor prior modifies start noise to reduce flicker.
Key results
- CCEdit achieves overall MOS 3.87 on a 1-5 scale, versus 3.58 for the closest competitor TokenFlow.
- In head-to-head comparisons CCEdit wins 52.9% vs TokenFlow (32.4% lose, 14.7% tie) and beats all eight baselines.
- Removing the edited key frame appearance control lowers MOS from 3.87 to 2.88.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CCEdit instead of TokenFlow | Video editing · mini-BalanceCC | editing accuracy (MOS): 3.78 -> 4.06 | Helped |
| Use CCEdit instead of TokenFlow | Video editing · mini-BalanceCC | aesthetics (MOS): 3.61 -> 4.00 | Helped |
| Use CCEdit instead of TokenFlow | Video editing · mini-BalanceCC | overall impression (MOS): 3.58 -> 3.87 | Helped |
| Use CCEdit instead of TokenFlow | Video editing · mini-BalanceCC | temporal consistency (MOS): 3.79 -> 3.74 | Hurt |
| Add an edited key-frame appearance reference (ablation) | CCEdit · BalanceCC user study | overall impression (MOS): 2.88 -> 3.87 | Helped |
| Add anchor prior to the start noise (ablation) | CCEdit · Video editing | temporal consistency | Helped |
Limitations
- Structure control relies on explicit structure extraction from the source video, so large structural changes (e.g., rabbit to tiger) are difficult.
- Flickering can occur when per-frame structure extraction is inconsistent, requiring the anchor prior.
- User study relies on 33 volunteers and 50 videos; automatic metrics are reported as not fully aligned with human preference.
Experimental setup
Datasets: WebVid-10M private video dataset
Benchmarks: BalanceCC (mini-BalanceCC subset of 50 videos)
Metrics: mean opinion score (edit accuracy, aesthetics, temporal cons win/tie/lose rate CLIP-Score (preliminary, in supplementary)
Baselines: Tune-A-Video vid2vid-zero Text2Video-Zero FateZero Pix2Video ControlVideo Rerender A Video TokenFlow
Builds on: Stable Diffusion ControlNet AnimateDiff VideoLDM DDIM
Closest papers by meaning
- VINCIE-NExT: Unlocking Video Editing from Images via In-Context Modeling (NeurIPS 2026 · similarity 0.88)
- Ctrl-Adapter: An Efficient and Versatile Framework for Adapting Diverse Controls to Any Diffusion Model (ICLR 2025 · similarity 0.88)
- FADE: Frequency-Aware Diffusion Model Factorization for Video Editing (CVPR 2025 · similarity 0.88)
- DiTCtrl: Exploring Attention Control in Multi-Modal Diffusion Transformer for Tuning-Free Multi-Prompt Longer Video Generation (CVPR 2025 · similarity 0.88)
- VideoDirector: Precise Video Editing via Text-to-Video Models (CVPR 2025 · similarity 0.87)
- Pathways on the Image Manifold: Image Editing via Video Generation (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.