Research map / Video / Video generation
Infinite-Canvas: Higher-Resolution Video Outpainting with Extensive Content Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Infinite-Canvas, a diffusion-based video outpainting method that splits the canvas into spatial windows merged with Gaussian weights, plus a layout encoder and relative region embedding for layout alignment, enabling high-resolution, high-expansion outpainting.
Problem
Existing diffusion-based video outpainting methods are limited to low resolutions and small content expansion ratios due to GPU memory constraints and degraded quality when outpainting large regions.
Approach
Distribute outpainting across overlapping spatial windows processed in parallel and merged with Gaussian weights at each denoising step, trained with randomly sampled anchor/target window pairs; inject the source video layout (SAM+Q-former layout encoder) and relative window position embedding via cross-attention to keep windows consistent with the global layout.
Key results
- Outpaints 512x512 to 2048x1152 (9x) and improves FVD from 928.6 (MOTIA) to 735.3 on DAVIS 2017.
- At 720P (~3.5x), FVD 440.0 vs 473.7 for best baseline MOTIA.
- Low-resolution 256x256 setting: FVD 242.8 vs 286.3 for MOTIA.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Infinite-Canvas instead of MOTIA | Video outpainting · DAVIS 2017, 1280 × 720 (720P, ∼3.5×) | FVD: 473.7 to 440.0 (+33.7) | Helped |
| Use Infinite-Canvas instead of MOTIA | Video outpainting · DAVIS 2017, 1440 × 810 (1.5K, ∼4.5×) | FVD: 575.9 to 486.1 (+89.8) | Helped |
| Use Infinite-Canvas instead of MOTIA | Video outpainting · DAVIS 2017, 2048 × 1152 (2K, 9×) | FVD: 928.6 to 735.3 (+193.3) | Helped |
| Use Infinite-Canvas instead of MOTIA | Video outpainting · Video outpainting to 256×256 | FVD: 286.3 to 242.8 | Helped |
| Add layout encoder and relative region embedding (ablation) | Infinite-Canvas · Video outpainting from 512×512 to 1440×810 | FVD: 774.1 -> 486.1 | Helped |
| Add layout encoder (ablation) | Infinite-Canvas with RRE · Video outpainting from 512×512 to 1440×810 | FVD: 512.2 -> 486.1 | Helped |
| Add relative region embedding (ablation) | Infinite-Canvas with LE · Video outpainting from 512×512 to 1440×810 | FVD: 670.3 -> 486.1 | Helped |
| Add layout extraction module (ablation) | Infinite-Canvas · Video outpainting from 512×512 to 1440×810 | FVD: 550.2 -> 486.1 | Helped |
| Replace CLIP image encoder with SAM encoder (ablation) | Infinite-Canvas layout encoder · Video outpainting from 512×512 to 1440×810 | FVD: 506.8 -> 486.1 | Helped |
Surprising findings
- Reducing resolution and increasing content expansion ratio both substantially degrade outpainting quality in existing methods.
Limitations
- Long inference time due to sliding-window inference over many windows.
- Requires multiple GPUs for practical acceleration.
Experimental setup
Datasets: Panda-70M DAVIS 2017
Benchmarks: DAVIS 2017
Metrics: FVD LPIPS AQ IQ PSNR SSIM
Baselines: M3DDM MOTIA Dehan et al. 2022 SDM
Builds on: AnimateDiff M3DDM MOTIA Latent diffusion models Gaussian diffusion (MultiDiffusion-style merging)
Closest papers by meaning
- Unboxed: Geometrically and Temporally Consistent Video Outpainting (CVPR 2025 · similarity 0.90)
- Elevating Flow-Guided Video Inpainting with Reference Generation (AAAI 2025 · similarity 0.88)
- Continuous-Multiple Image Outpainting in One-Step via Positional Query and A Diffusion-based Approach (ICLR 2024 · similarity 0.87)
- Keyframe-Guided Creative Video Inpainting (CVPR 2025 · similarity 0.87)
- Video Diffusion Models Are Strong Video Inpainter (AAAI 2025 · similarity 0.86)
- Frame In-N-Out: Unbounded Controllable Image-to-Video Generation (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.