Research map / 3D vision / Gaussian splatting
Generative Sparse-View Gaussian Splatting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GS-GS, a pipeline that jointly optimizes 3D/4D Gaussian Splatting and a LoRA-adapted depth-conditioned Stable Diffusion model that hallucinates pseudo-view images, with a geometry-aware feature loss and MS-SSIM depth regularization, for sparse-view reconstruction.
Problem
Novel view synthesis from sparse training views leads to artifacts and geometric ambiguity in under-sampled regions for 3D and 4D Gaussian Splatting.
Approach
Alternately fine-tune a LoRA-adapted pre-trained diffusion model (with depth T2I-adapter) on rendered views and use it to hallucinate pseudo-view training images for Gaussian Splatting, enforcing cross-view feature consistency via warping and adding a monocular-depth MS-SSIM regularizer.
Key results
- On Blender (8 views), GS-GS reports PSNR 28.57, SSIM 0.923, LPIPS 0.055, over 3 dB above the second-best FSGS.
- On LLFF (3 views), GS-GS reaches PSNR 24.82 vs FSGS 20.31; on Mip-NeRF360 it reaches PSNR 25.87 vs FSGS 23.70.
- On Neural 3D Video with 3 views, GS-GS reaches PSNR 27.13 vs SpacetimeGS 14.98.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add diffusion hallucination (ablation) | GS-GS based on 3DGS · LLFF | PSNR: 17.43 -> 22.71 | Helped |
| add diffusion hallucination (ablation) | GS-GS based on 3DGS · Mip-NeRF360 | PSNR: 20.89 -> 23.23 | Helped |
| add geometry-aware fine-tuning (ablation) | GS-GS based on 3DGS · LLFF | PSNR: 22.71 -> 24.09 | Helped |
| add geometry-aware fine-tuning (ablation) | GS-GS based on 3DGS · Mip-NeRF360 | PSNR: 23.23 -> 25.28 | Helped |
| add depth regularization (ablation) | GS-GS based on 3DGS · LLFF | PSNR: 24.09 -> 24.82 | Helped |
| add depth regularization (ablation) | GS-GS based on 3DGS · Mip-NeRF360 | PSNR: 25.28 -> 25.87 | Helped |
Limitations
- Generated pseudo-view images remain not perfectly geometry-consistent, causing some blur in rendered results.
- Pipeline relies on monocular depth estimates (DPT) for depth regularization and diffusion conditioning.
Experimental setup
Datasets: NeRF Blender Synthetic LLFF Mip-NeRF360 Neural 3D Video
Benchmarks: NeRF Blender Synthetic LLFF Mip-NeRF360 Neural 3D Video
Metrics: PSNR SSIM LPIPS
Baselines: Mip-NeRF 3DGS DietNeRF RegNeRF FreeNeRF SparseNeRF DNGaussian FSGS SpacetimeGS
Builds on: 3D Gaussian Splatting SpacetimeGS Stable Diffusion T2I-adapter DIFT
Closest papers by meaning
- DropGaussian: Structural Regularization for Sparse-view Gaussian Splatting (CVPR 2025 · similarity 0.93)
- FewViewGS: Gaussian Splatting with Few View Matching and Multi-stage Training (NeurIPS 2024 · similarity 0.93)
- Speedy-Splat: Fast 3D Gaussian Splatting with Sparse Pixels and Sparse Primitives (CVPR 2025 · similarity 0.92)
- Quantifying and Alleviating Co-Adaptation in Sparse-View 3D Gaussian Splatting (NeurIPS 2025 · similarity 0.92)
- 3DGS-Enhancer: Enhancing Unbounded 3D Gaussian Splatting with View-consistent 2D Diffusion Priors (NeurIPS 2024 · similarity 0.92)
- GaussianSpa: An "Optimizing-Sparsifying" Simplification Framework for Compact and High-Quality 3D Gaussian Splatting (CVPR 2025 · similarity 0.92)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.