Research map / Video / Video generation
PISCO: Precise Video Instance Insertion with Sparse Control
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PISCO, a Wan-based video diffusion model that inserts a user-specified instance into a clean video under arbitrary sparse keyframe control, with a new benchmark PISCO-Bench built from BURST.
Problem
Video instance insertion needs precise spatial-temporal placement and physically consistent effects, but existing inpainting, V2V editing, agentic image-edit+I2V, and 4D reconstruction methods require dense per-frame masks, drift from the background, or are computationally heavy.
Approach
Conditions a pretrained Wan VACE adapter on sparse instance RGB, mask, depth and an availability signal, using Variable-Information Guidance (random availability masks during training), Distribution-Preserving Temporal Masking (pixel-space nearest-frame fill plus token-space masking), amodal and relighting augmentations.
Key results
- PISCO-14B (First & Last) reduces whole-video FVD from 371 (VACE 14B) to 204 and foreground FVD to 138.
- PISCO-14B with five control frames reaches whole-video FVD 136 and foreground LPIPS 0.015.
- In VBench, PISCO-14B (First & Last) achieves Subject Consistency 91.57 vs. 90.29 for VACE 14B.
Limitations
- Baselines lack a comparable sparse-control interface, so comparisons use different pipelines and conditions.
- First-frame-only control lets trajectory drift over time.
- Evaluation set is only 100 videos from BURST.
Experimental setup
Datasets: ROSE VPData MOSE DAVIS BURST
Benchmarks: PISCO-Bench
Metrics: FVD LPIPS PSNR SSIM VBench background consistency VBench subject consistency aesthetic quality imaging quality motion smoothness temporal flickering
Baselines: Nano-banana-Pro + Wan2.2-Fun-A14B-InP CoCoCo VideoPainter VACE UniVideo
Builds on: Wan2.1 VACE Wan2.2 IC-Light Depth Anything V3
Closest papers by meaning
- Frame In-N-Out: Unbounded Controllable Image-to-Video Generation (NeurIPS 2025 · similarity 0.86)
- MotionPro: A Precise Motion Controller for Image-to-Video Generation (CVPR 2025 · similarity 0.85)
- Generative Inbetweening: Adapting Image-to-Video Models for Keyframe Interpolation (ICLR 2025 · similarity 0.85)
- Motion Prompting: Controlling Video Generation with Motion Trajectories (CVPR 2025 · similarity 0.85)
- Through-The-Mask: Mask-based Motion Trajectories for Image-to-Video Generation (CVPR 2025 · similarity 0.85)
- PEEKABOO: Interactive Video Generation via Masked-Diffusion (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.