Research map / Video / Video generation
Unboxed: Geometrically and Temporally Consistent Video Outpainting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a video outpainting pipeline that combines 3D Gaussian Splatting for static background outpainting with per-frame object inpainting and a pre-trained video diffusion model applied as a guided denoising filter for temporal consistency.
Problem
Extending the field of view of videos (outpainting) while keeping outpainted content geometrically and temporally consistent across frames, which existing generative-only video diffusion methods handle poorly in quality and memory.
Approach
Decompose the video into static and dynamic regions: outpaint static regions by iteratively alternating SDXL image outpainting with updating a 3D Gaussian Splatting scene model (with depth losses), inpaint dynamic objects per frame with a fine-tuned SDXL, then blend known latents into a pre-trained image-to-video diffusion denoising process with a mask so the video model acts as a temporal filter.
Key results
- On DAVIS, FVD improves to 232.5 from 286.3 (MOTIA paper) / 294.8 (MOTIA model), with Ewarp 0.015 and PSNR 22.62, the best among compared methods.
- On YouTube-VOS, the method reaches FVD 114.2 and Ewarp 0.016, improving FVD by 26.9% over the best baseline.
- User study with 619 votes from 37 participants preferred the method at over 80% in all dimensions.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the full 3D-supported and guided-video-synthesis method rather than MOTIA (paper) | Video outpainting · DAVIS | FVD: 286.3 -> 232.5 | Helped |
| Use the full method rather than M3DDM | Video outpainting · YouTube-VOS | Ewarp: 0.024 -> 0.016 | Helped |
| Use the full method rather than MOTIA (model) | Video outpainting · YouTube-VOS | FVD: 156.4 -> 114.2 | Helped |
| Add static-region GS outpainting, guided video synthesis, and object-inpainting initialization instead of using only fine-tuned image-model outpainting (ablation) | Video outpainting on DAVIS · DAVIS | FVD: 448.8 -> 232.5 | Helped |
| Add guided video synthesis to fine-tuned image-model outpainting (ablation) | Video outpainting on DAVIS · DAVIS | Ewarp: 0.032 -> 0.022 | Helped |
| Add guided video synthesis (ablation) | Video outpainting on DAVIS · DAVIS | FVD: 293.3 -> 232.5 | Helped |
| Add object-inpainting initialization (ablation) | Video outpainting on DAVIS · DAVIS | Ewarp: 0.017 -> 0.015 | Helped |
Surprising findings
- Ablation shows that applying video diffusion guided synthesis alone to independently outpainted frames does not fix coherence between outpainted and original content.
Limitations
- Objects moving in and out of the original frame remain challenging; bounding-box trajectory extrapolation is limited.
- Multiple occluded dynamic objects are handled by independent inpainting and depth-based blending, which is limited.
- Metrics like PSNR, SSIM and LPIPS are acknowledged as limited in capturing outpainting quality.
Experimental setup
Datasets: DAVIS YouTube-VOS
Benchmarks: DAVIS YouTube-VOS
Metrics: PSNR SSIM LPIPS FVD flow warping error (Ewarp)
Baselines: Dehan et al. M3DDM MOTIA
Builds on: 3D Gaussian Splatting ReaFill RePaint Stable Video Diffusion AnimateDiff
Closest papers by meaning
- Infinite-Canvas: Higher-Resolution Video Outpainting with Extensive Content Generation (AAAI 2025 · similarity 0.90)
- Reconstruct, Inpaint, Test-Time Finetune: Dynamic Novel-view Synthesis from Monocular Videos (NeurIPS 2025 · similarity 0.88)
- Continuous-Multiple Image Outpainting in One-Step via Positional Query and A Diffusion-based Approach (ICLR 2024 · similarity 0.88)
- Generative Omnimatte: Learning to Decompose Video into Layers (CVPR 2025 · similarity 0.87)
- InstaInpaint: Instant 3D-Scene Inpainting with Masked Large Reconstruction Model (NeurIPS 2025 · similarity 0.86)
- ViVid-1-to-3: Novel View Synthesis with Video Diffusion Models (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.