Research map / 3D vision / Gaussian splatting
Novel View Synthesis from A Few Glimpses via Test-Time Natural Video Completion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Recasts sparse-input novel view synthesis as test-time natural video completion, using a frozen pretrained video diffusion model (Stable Video Diffusion) with uncertainty-aware guidance to generate pseudo views that densify 3D Gaussian Splatting training, iterated with reconstruction.
Problem
Sparse-input novel view synthesis with 3D-GS suffers from under-constrained occluded and out-of-view regions when only a few widely spaced input views are available.
Approach
Zero-shot, generation-guided 3D-GS: guidance images are built by inverse-warping nearest input views using 3D-GS depth, their cross-view uncertainty modulates the video diffusion denoising (SDEdit-like fusion of predicted clean latents), and generated pseudo views both supervise and densify Gaussians in an iterative loop.</1antml:parameter>
Key results
- On DL3DV with 3 views, PSNR 14.62 vs 11.70 for the next best (IPSM); 19.19 PSNR with 9 views.
- On DTU (3 views) reaches 20.51 PSNR, 0.840 SSIM, 0.137 LPIPS, outperforming the next-best method.
- On MipNeRF-360 (9 views) reaches 17.91 PSNR vs 16.68 for ViewCrafter.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use video-diffusion pseudo-view supervision and uncertainty-aware warping guidance | 3D-GS · DL3DV (3 views) | PSNR: 12.22 -> 14.62 | Helped |
| Use video-diffusion pseudo-view supervision and uncertainty-aware warping guidance | 3D-GS · DL3DV (6 views) | PSNR: 13.73 -> 17.35 | Helped |
| Use video-diffusion pseudo-view supervision and uncertainty-aware warping guidance | 3D-GS · DL3DV (9 views) | PSNR: 15.52 -> 19.19 | Helped |
| Use warping-based guidance instead of 3D-GS rendered guidance | 3D-GS pseudo-view generation · DL3DV test set | PSNR: 18.59 -> 19.19 | Helped |
| Add geometric consistency to uncertainty estimation (ablation) | 3D-GS pseudo-view generation · DL3DV test set | PSNR: 18.21 -> 19.19 | Helped |
| Add photometric consistency to uncertainty estimation (ablation) | 3D-GS pseudo-view generation · DL3DV test set | PSNR: 18.93 -> 19.19 | Helped |
| Add point filtering (ablation) | 3D-GS optimization · DL3DV test set | PSNR: 19.01 -> 19.19 | Helped |
| Add 3D-GS densification (ablation) | 3D-GS optimization · DL3DV test set | PSNR: 18.23 -> 19.19 | Helped |
| Use LPIPS instead of L1 loss for pseudo-view supervision (ablation) | 3D-GS optimization · DL3DV test set | PSNR: 18.81 -> 19.19 | Helped |
Surprising findings
- Image-based diffusion (IPSM) struggles at extremely sparse inputs, while video diffusion infers scene context from the reference frame.
Limitations
- Depends on quality of the pretrained video diffusion model, which can introduce artifacts under extreme views or complex scenes.
- Iterative training adds overhead over vanilla 3D-GS pipelines.
- Early 3D-GS depth errors can degrade guidance quality.
Experimental setup
Datasets: LLFF DL3DV DTU MipNeRF-360
Benchmarks: LLFF DTU DL3DV MipNeRF-360
Metrics: PSNR SSIM LPIPS
Baselines: Mip-NeRF 3D-GS DietNeRF RegNeRF FreeNeRF SparseNeRF FSGS DNGaussian IPSM SparseGS MVSplat 360 ViewCrafter
Builds on: 3D Gaussian Splatting Stable Video Diffusion SDEdit FSGS DNGaussian
Closest papers by meaning
- MVSplat360: Feed-Forward 360 Scene Synthesis from Sparse Views (NeurIPS 2024 · similarity 0.92)
- Generative Sparse-View Gaussian Splatting (CVPR 2025 · similarity 0.91)
- Taming Video Diffusion Prior with Scene-Grounding Guidance for 3D Gaussian Splatting from Sparse Inputs (CVPR 2025 · similarity 0.90)
- NAMVIS: Next-Scale Autoregressive Multi-View Image Synthesis (NeurIPS 2026 · similarity 0.89)
- UMAMI: Unifying Masked Autoregressive Models and Deterministic Rendering for View Synthesis (NeurIPS 2025 · similarity 0.89)
- Sparse3D: Distilling Multiview-Consistent Diffusion for Object Reconstruction from Sparse Views (AAAI 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.