Research map / 3D vision / Gaussian splatting
Structure Consistent Gaussian Splatting with Matching Prior for Few-shot Novel View Synthesis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SCGaussian, a few-shot 3D Gaussian Splatting method that uses pretrained matching priors and a hybrid of ray-bound and free Gaussian primitives to enforce a 3D-consistent scene structure.
Problem
Few-shot novel view synthesis with 3D Gaussian Splatting fails under sparse inputs because the model cannot learn a 3D-consistent scene structure; Gaussian positions are hard to constrain and monocular depth priors are scale- and multi-view inconsistent.
Approach
A hybrid Gaussian representation: ray-based primitives bound to matching rays (position restricted to move along the ray, supervised by matching correspondences) plus ordinary non-structure primitives for single-view background, together with a rendering-geometry reprojection loss.
Key results
- On LLFF with 3 views, SCGaussian reaches 20.77 PSNR / 0.705 SSIM / 0.218 LPIPS, beating FSGS (20.43 PSNR) and DNGaussian (19.12 PSNR).
- On Tanks and Temples with 3 views, SCGaussian achieves 22.17 PSNR vs. 20.01 for FSGS, a gain of roughly 2 dB.
- Training converges within 1 minute on a single RTX 3090 (~3GB memory) with over 200 FPS rendering at 504x378 resolution.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add hybrid Gaussian representation to vanilla 3DGS (ablation) | 3DGS · LLFF, 3 views | PSNR: 16.46 -> 18.62 | Helped |
| add rendering geometry optimization (ablation) | 3DGS with hybrid representation · LLFF, 3 views | PSNR: 18.62 -> 19.40 | Helped |
| add dual optimization of rendering geometry and Gaussian positions (ablation) | SCGaussian with hybrid representation · LLFF, 3 views | PSNR: 19.40 -> 20.69 | Helped |
| add cache and matching-prior filtering (ablation) | SCGaussian with dual optimization · LLFF, 3 views | PSNR: 20.69 -> 20.77 | Helped |
| replace non-structure primitives with ray-based primitives only (ablation) | SCGaussian hybrid representation · LLFF, 3 views | PSNR: 20.77 -> 20.50 | Hurt |
| replace ray-based primitives with non-structure primitives only (ablation) | SCGaussian hybrid representation · LLFF, 3 views | PSNR: 20.77 -> 19.40 | Hurt |
| use FSGS instead of SCGaussian | few-shot novel view synthesis · LLFF, 3 views | PSNR: 20.43 -> 20.77 | Hurt |
| use FSGS instead of SCGaussian | few-shot novel view synthesis · IBRNet, 3 views | PSNR: 19.84 -> 21.59 | Hurt |
| use FSGS instead of SCGaussian | few-shot novel view synthesis · Tanks and Temples, 3 views | PSNR: 20.01 -> 22.17 | Hurt |
| use FreeNeRF instead of SCGaussian | few-shot novel view synthesis · DTU, 3 views | PSNR: 19.92 -> 20.56 | Hurt |
| use FSGS instead of SCGaussian | few-shot novel view synthesis · Blender, 8 training images | PSNR: 24.640 -> 25.618 | Hurt |
| increase training views from 3 to 6 | SCGaussian · Tanks and Temples | PSNR: 22.17 -> 26.95 | Helped |
Surprising findings
- DNGaussian, which uses monocular depth regularization, performs worse than vanilla 3DGS on the complex T&T scenes, which the authors attribute to scale and multi-view inconsistency of monocular depth.
Limitations
- Requires accurate camera poses, which may not be available in practice.
- Relies on the quality of pretrained matching models; mismatched ray pairs require filtering.
Experimental setup
Datasets: LLFF IBRNet Tanks and Temples DTU NeRF Blender Synthetic
Benchmarks: LLFF IBRNet Tanks and Temples DTU NeRF Blender Synthetic
Metrics: PSNR SSIM LPIPS AVG
Baselines: MipNeRF DietNeRF RegNeRF FreeNeRF SparseNeRF 3DGS FSGS DNGaussian
Builds on: 3D Gaussian Splatting DNGaussian FSGS GIM
Closest papers by meaning
- FewViewGS: Gaussian Splatting with Few View Matching and Multi-stage Training (NeurIPS 2024 · similarity 0.94)
- COLMAP-Free 3D Gaussian Splatting (CVPR 2024 · similarity 0.93)
- GaussianPro: 3D Gaussian Splatting with Progressive Propagation (ICML 2024 · similarity 0.92)
- Generative Sparse-View Gaussian Splatting (CVPR 2025 · similarity 0.92)
- DropGaussian: Structural Regularization for Sparse-view Gaussian Splatting (CVPR 2025 · similarity 0.92)
- MaskGaussian: Adaptive 3D Gaussian Representation from Probabilistic Masks (CVPR 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.