Research map / 3D vision / Gaussian splatting
SOVGaussian: Sparse-View 3D Gaussian Splatting for Open-Vocabulary Scene Understanding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SOVGaussian, a sparse-view (3-view) 3D Gaussian Splatting method that builds a depth-constrained language field supporting open-vocabulary querying in novel views.
Problem
Open-vocabulary language fields in 3D Gaussian splatting require dense training views and fail to answer language queries from novel views when only a few input images are available.
Approach
Combines a monocular depth prior distilled per SAM object mask (Language-Aware Depth Distillation) with language-guided outlier pruning that uses LOF on primitives grouped by their language prototype, so that language and geometry constrain each other under sparse views.
Key results
- On 3DOVS with 3 training views, SOVGaussian reaches 65.9% overall mIoU in novel views, versus 42.0% for LangSplat and 32.4% for LEGaussians.
- On DTU's 7 annotated scenes, overall mIoU is 71.7% versus 52.7% for LangSplat.
- Ablation removing the depth loss drops mIoU from 65.9 to 46.2 and PSNR from 17.63 to 15.00.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SOVGaussian instead of LangSplat | 3D open-vocabulary scene querying · 3DOVS novel views | mIoU: 42.0 -> 65.9 | Helped |
| Use SOVGaussian instead of LangSplat | 3D open-vocabulary scene querying · DTU novel views | mIoU: 52.7 -> 71.7 | Helped |
| Add Language-Guided Outlier Pruning (ablation) | SOVGaussian · 3DOVS novel views | mIoU: 61.8 -> 65.9 | Helped |
| Add Language-Aware Depth Distillation (ablation) | SOVGaussian · 3DOVS novel views | mIoU: 63.4 -> 65.9 | Helped |
| Add depth loss (ablation) | SOVGaussian · 3DOVS novel views | mIoU: 46.2 -> 65.9 | Helped |
| Use language hash grids instead of 3-dimensional embeddings (ablation) | SOVGaussian · 3DOVS novel views | mIoU: 64.5 -> 65.9 | Helped |
Surprising findings
- Quantizing the language embedding to a low-dimensional index performs poorly under sparse views, so the authors use a continuous 3-dimensional autoencoder embedding instead.
Limitations
- Relies on static primitives and cannot handle dynamic scenes.
- Monocular depth estimates are numerically inaccurate at global scale, so depth is used only for relative ordering within masks.
- DTU open-vocabulary annotations were created manually by the authors for 7 scenes, which is a small evaluation set.
Experimental setup
Datasets: 3DOVS DTU
Benchmarks: 3DOVS DTU
Metrics: mIoU localization accuracy (mAcc) PSNR SSIM
Baselines: 3DOVS LEGaussians LangSplat
Builds on: 3D Gaussian Splatting LangSplat hard and soft depth regularization Local Outlier Factor hash grid
Closest papers by meaning
- Language Embedded 3D Gaussians for Open-Vocabulary Scene Understanding (CVPR 2024 · similarity 0.92)
- LangSplat: 3D Language Gaussian Splatting (CVPR 2024 · similarity 0.91)
- Tackling View-Dependent Semantics in 3D Language Gaussian Splatting (ICML 2025 · similarity 0.90)
- OpenGaussian: Towards Point-Level 3D Gaussian-based Open Vocabulary Understanding (NeurIPS 2024 · similarity 0.89)
- LangSplatV2: High-dimensional 3D Language Gaussian Splatting with 450+ FPS (NeurIPS 2025 · similarity 0.89)
- On the Robustness of Language Guidance for Low-Level Vision Tasks: Findings from Depth Estimation (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.