Research map / 3D vision / Gaussian splatting
Tackling View-Dependent Semantics in 3D Language Gaussian Splatting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LaGa, which decomposes a 3D-GS scene into objects via contrastive affinity features and aggregates multi-view CLIP semantics into weighted per-object descriptors for direct 3D open-vocabulary segmentation.
Problem
Language-embedded 3D Gaussian Splatting methods lift 2D CLIP features onto 3D Gaussians, but these features are view-dependent, so direct 3D open-vocabulary retrieval yields false positives and false negatives.
Approach
Use SAM-mask-based contrastive 3D scene decomposition to link multi-view masks of the same object, then cluster each object's multi-view CLIP features into adaptive K-means descriptors and weight them by directional consistency with the object's global feature and by cluster compactness (descriptor norm).
Key results
- On LERF-OVS, LaGa reaches 64.0% mean mIoU, +18.7% over the reimplemented 3D baseline OpenGaussian (45.3%) under identical settings.
- LaGa surpasses the 2D SOTA VLGS (62.0%) by +2.0 points and the abstract claims +8.8% over 2D SOTA on LERF-OVS (per the paper's comparison).
- On 3D-OVS LaGa reaches 95.3% mIoU, comparable to 2D methods (VLGS 97.1%, OccamLGS 95.0%), and on ScanNet (19 classes) it reaches 32.5 mIoU vs 24.7 for OpenGaussian.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LaGa's view-aggregated semantic representation instead of OpenGaussian's rigid feature assignment | 3D open-vocabulary segmentation · LERF-OVS | mIoU: +18.7% mIoU | Helped |
| Use LaGa instead of VLGS | 3D open-vocabulary segmentation · 3D-OVS | mIoU: 97.1 -> 95.3 | Hurt |
| Use LaGa instead of OpenGaussian | 3D point-cloud segmentation · ScanNet (19 classes) | mIoU: 24.7 -> 32.5 | Helped |
| Replace adaptive descriptor clustering with average pooling (ablation) | LaGa · LERF-OVS | mIoU: 10.6% mIoU | Hurt |
| Replace adaptive descriptor clustering with max pooling (ablation) | LaGa · LERF-OVS | mIoU: 20.7% mIoU | Hurt |
| Add descriptor weighting to fixed 5-cluster extraction (ablation) | LaGa · LERF-OVS | mIoU: 59.1 -> 61.9 | Helped |
| Add descriptor weighting to fixed 10-cluster extraction (ablation) | LaGa · LERF-OVS | mIoU: 59.4 -> 62.3 | Helped |
| Add descriptor weighting to fixed 20-cluster extraction (ablation) | LaGa · LERF-OVS | mIoU: 57.3 -> 59.2 | Helped |
| Add full descriptor weighting to adaptive clustering (ablation) | LaGa · LERF-OVS | mIoU: 60.4 -> 64.0 | Helped |
| Use adaptive clustering instead of 20 fixed clusters with descriptor weighting (ablation) | LaGa · LERF-OVS | mIoU: 59.2 -> 64.0 | Helped |
| Use compactness-only descriptor weighting instead of no weighting (ablation) | LaGa with adaptive clustering · LERF-OVS | mIoU: 60.4 -> 60.9 | Helped |
| Use alignment-only descriptor weighting instead of no weighting (ablation) | LaGa with adaptive clustering · LERF-OVS | mIoU: 60.4 -> 61.9 | Helped |
Surprising findings
- Naive max pooling over multi-view features drops mIoU by about 20.7 points relative to the adaptive approach, showing that retaining all view features is harmful.
- On 3D-OVS, LaGa does not significantly outperform 2D methods, which the authors attribute to dataset simplicity and forward-facing views.
Limitations
- Inherits CLIP's bag-of-words limitation for compositional queries.
- Cropping masks for CLIP removes context, causing errors on relational or part-level queries.
- Needle-like borders from ambiguous 3D Gaussians remain, which the paper leaves out of scope.
Experimental setup
Datasets: LERF-OVS 3D-OVS ScanNet MIP-360
Benchmarks: LERF-OVS 3D-OVS ScanNet
Metrics: mIoU mAcc
Baselines: LSeg LERF LEGaussians LangSplat N2F2 OccamLGS VLGS OpenGaussian SAGA SuperGSeg 3D-OVS GOI
Builds on: 3D Gaussian Splatting LangSplat OpenGaussian SAGA HDBSCAN
Closest papers by meaning
- Bootstraping Clustering of Gaussians for View-consistent 3D Scene Understanding (AAAI 2025 · similarity 0.92)
- 3D Vision-Language Gaussian Splatting (ICLR 2025 · similarity 0.91)
- Language Embedded 3D Gaussians for Open-Vocabulary Scene Understanding (CVPR 2024 · similarity 0.91)
- Dr. Splat: Directly Referring 3D Gaussian Splatting via Direct Language Embedding Registration (CVPR 2025 · similarity 0.91)
- Chain of Semantics Programming in 3D Gaussian Splatting Representation for 3D Vision Grounding (CVPR 2025 · similarity 0.91)
- SceneSplat++: A Large Dataset and Comprehensive Benchmark for Language Gaussian Splatting (NeurIPS 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.