Research map / Video / Video generation
Controllable First-Frame-Guided Video Editing via Mask-Aware LoRA Fine-Tuning
Chenjian Gao, Lihe Ding, Xin Cai, Zhanpeng Huang, Zibin Wang, Tianfan Xue
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use input-level mask-aware LoRA training instead of feature-level latent blending (ablation) | mask-aware LoRA editing · first-frame-guided editing | CLIP Score: 0.8936 -> 0.9172 | Improved |
| use input-level mask-aware LoRA training instead of feature-level latent blending (ablation) | mask-aware LoRA editing · first-frame-guided editing | DeQA Score: 3.5878 -> 3.8013 | Improved |
| use input-level mask-aware LoRA training instead of feature-level latent blending (ablation) | mask-aware LoRA editing · first-frame-guided editing | Input Similarity: 0.7402 -> 0.7608 | Improved |
| add foreground-background mask conditioning (ablation) | mask-aware LoRA editing · hair color and clothing edits | qualitative | Improved |
| add an edited later frame as appearance reference (ablation) | mask-aware LoRA editing · propagated edits | qualitative appearance control | Improved |
| use loose bounding-box or noisy mask instead of tight SAM2 mask (ablation) | mask-aware LoRA editing · mask robustness | qualitative | Improved |
| use mask scheduling in appearance-learning stage instead of default I2V mask (ablation) | mask-aware LoRA editing · appearance learning | qualitative | Improved |
| train on 9-frame sliding windows with swap blocks instead of full 49 frames (ablation) | mask-aware LoRA editing · memory | peak VRAM (MB): 21,522 -> 7,617 | Improved |
| increase blocks_to_swap from 0 to 38 (ablation) | mask-aware LoRA editing · peak VRAM (9-frame segments) | peak VRAM (MB): 25,097 -> 7,617 | Improved |
| vary clip length 5, 13, 21, 49 frames (ablation) | mask-aware LoRA editing · CLIP Score vs frame count | CLIP Score | No clear effect |
| compare with Kling1.6 on user ranking | mask-aware LoRA editing · reference-guided editing user study | motion consistency rank: 1.869 -> 1.620 | Improved |
| text and rapid abstract motion | mask-aware LoRA editing · failure cases | qualitative | Hurt |
Closest papers by meaning
- Transforming Image Editors into Video Editors (NeurIPS 2026 · similarity 0.87)
- MaskINT: Video Editing via Interpolative Non-autoregressive Masked Transformers (CVPR 2024 · similarity 0.87)
- VINCIE-NExT: Unlocking Video Editing from Images via In-Context Modeling (NeurIPS 2026 · similarity 0.87)
- FreeMask: Rethinking the Importance of Attention Masks for Zero-Shot Video Editing (AAAI 2025 · similarity 0.87)
- VideoDirector: Precise Video Editing via Text-to-Video Models (CVPR 2025 · similarity 0.87)
- ReVideo: Remake a Video with Motion and Content Control (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
