Research map / 3D vision / Gaussian splatting
SPC-GS: Gaussian Splatting with Semantic-Prompt Consistency for Indoor Open-World Free-view Synthesis from Sparse Inputs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SPC-GS, a 3DGS framework for indoor open-world free-view synthesis and open-vocabulary segmentation from 12 sparse views, using video-generated views for denser scene-layout Gaussian initialization and SAM2-based semantic-prompt consistency regularization.
Problem
Open-world semantic 3D Gaussian splatting for indoor scenes degrades sharply with sparse input views because SfM initialization yields few sparse Gaussians and training views give insufficient supervision.
Approach
Generate adjacent views with a video generation model (MotionCtrl) to densify SfM initialization into scene-layout Gaussians, then use SAM2 region masks propagated from training views as semantic prompts to enforce 2D and 3D semantic consistency on rendered pseudo views.
Key results
- On Replica with 12 views, SPC-GS reaches 22.011 PSNR and 23.960 mIoU, versus 18.955 PSNR and 16.657 mIoU for the best prior baselines.
- On ScanNet, SPC-GS reaches 22.401 PSNR and 28.944 mIoU, outperforming Gau-Grouping's 19.006 PSNR and 15.172 mIoU.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add VGM-based Point Creation (PC) (ablation) | SPC-GS · Replica | PSNR: 17.134 -> 18.924 | Helped |
| Add Scene-layout Point Generation (SPG) after PC (ablation) | SPC-GS · Replica | PSNR: 18.924 -> 21.176 | Helped |
| Add 2D SPC constraints (ablation) | SPC-GS with SGI · Replica | mIoU: 19.401 -> 22.849 | Helped |
| Add 3D SPC constraints (ablation) | SPC-GS with SGI and 2D SPC · Replica | mIoU: 22.849 -> 23.960 | Helped |
| Add view-to-view regularization (ablation) | SPC-GS · Replica | mIoU: 21.244 -> 23.960 | Helped |
| Add contrastive regularization (ablation) | SPC-GS · Replica | mIoU: 22.322 -> 23.960 | Helped |
| Add region boundary erosion (ablation) | SPC-GS · Replica | mIoU: 22.964 -> 23.960 | Helped |
| Use SGI instead of sparse SfM point initialization (ablation) | SPC-GS · Replica | mIoU: 14.899 -> 18.169 | Helped |
| Add SPC to SGI (ablation) | SPC-GS · Replica | mIoU: 18.169 -> 20.766 | Helped |
Surprising findings
- Using generated video frames for color supervision degraded results, so they are used only for semantic loss.
Limitations
- Relies on a video generation model to create additional views, adding dependence on generative model quality.
- Evaluated only on indoor Replica and ScanNet scenes with 12 training views.
Experimental setup
Datasets: Replica ScanNet
Benchmarks: Replica ScanNet
Metrics: PSNR SSIM LPIPS mIoU mAcc
Baselines: 3DOVS Feature 3DGS LangSplat Gau-Grouping DNGaussian FSGS CoR-GS
Builds on: 3D Gaussian Splatting Feature 3DGS LangSplat CoR-GS FSGS MotionCtrl
Closest papers by meaning
- FreeSplat: Generalizable 3D Gaussian Splatting Towards Free View Synthesis of Indoor Scenes (NeurIPS 2024 · similarity 0.91)
- Generative Sparse-View Gaussian Splatting (CVPR 2025 · similarity 0.89)
- FewViewGS: Gaussian Splatting with Few View Matching and Multi-stage Training (NeurIPS 2024 · similarity 0.89)
- Binocular-Guided 3D Gaussian Splatting with View Consistency for Sparse View Synthesis (NeurIPS 2024 · similarity 0.88)
- Structure Consistent Gaussian Splatting with Matching Prior for Few-shot Novel View Synthesis (NeurIPS 2024 · similarity 0.88)
- SceneSplat++: A Large Dataset and Comprehensive Benchmark for Language Gaussian Splatting (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.