Research map / Vision-language and multimodal / Multimodal LLMs

VicEdit: Learning to Edit Videos from Visual In-Context Examples

NeurIPS 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Interleaved image-text generation with MLLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes visual in-context video editing with single-image, image-pair, and video-pair references, a 400K-sample dataset (VicEdit-400K), and the VicEdit framework using modality-adaptive semantic distillation and dual-context injection on a Wan video DiT.

Problem

Text-only instructions cannot convey fine-grained textures, spatial transformations, or complex motion for video editing, and existing reference-based editing methods support only one reference modality and lack large multimodal training data.

Approach

Modality-conditioned learnable queries (base queries plus modality-specific embeddings) distill editing semantics from heterogeneous visual references, and a semantic-shift MLP lets the visual context calibrate instruction queries before both token sets are injected into a video DiT via cross-attention.

Key results

• VicEdit reaches Overall 3.45 on VicEdit-Bench versus 2.97 for the second-best Kiwi-Edit (about 16% relative gain).
• On OpenVE-Ext base editing, VicEdit achieves Overall 3.28, the best among six baselines, with Kiwi-Edit at 3.08.
• Ablation: adding MASD and DCI to the base raises VLM Overall from 3.02 to 3.25 (MASD) and 3.34 (DCI), and 3.45 with both.

Surprising findings

• Stage-2 in-context training improves VIC metrics substantially but lowers base editing performance (3.23 to 3.10), indicating a trade-off between task types.

Limitations

• Current design targets semantic-level editing and not pixel-level precision.
• Baselines on VicEdit-Bench are given an instruction enhancement module, and the evaluation relies largely on MLLM-based scoring.
• Dataset scale is limited to 400K samples and future work is needed to scale further.

Experimental setup

Datasets: VicEdit-400K Señorita-2M OpenVE-3M OpenVE-Bench

Benchmarks: VicEdit-Bench OpenVE-Ext VBench

Metrics: overall score (MLLM-judged 1-5) instruction compliance detail fidelity visual quality VBench background consistency motion smoothness temporal flickering imaging quality aesthetic quality

Baselines: VACE Lucy-Edit DITTO NovaEdit VideoCoF Kiwi-Edit

Builds on: Kiwi-Edit Wan2.2-TI2V-5B OpenVE Señorita-2M

Closest papers by meaning

• Text-Vision Co-Instructed Image Editing (NeurIPS 2026 · similarity 0.90)
• VINCIE-NExT: Unlocking Video Editing from Images via In-Context Modeling (NeurIPS 2026 · similarity 0.89)
• Señorita-2M: A High-Quality Instruction-based Dataset for General Video Editing by Video Specialists (NeurIPS 2025 · similarity 0.88)
• Edit-R2: Context-Aware Reinforcement Learning for Multi-Turn Image Editing (NeurIPS 2026 · similarity 0.88)
• SteerVTE: Seamless Video Text Editing with Style and Glyph Control (NeurIPS 2026 · similarity 0.87)
• InsightEdit: Towards Better Instruction Following for Image Editing (CVPR 2025 · similarity 0.87)