Research map / 3D vision / Gaussian splatting
Segment Any 3D Gaussians
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
SAGA attaches scale-gated affinity features to 3D Gaussians, distilled from SAM masks via scale-aware contrastive learning, enabling real-time (~4 ms) 3D promptable segmentation in 3D-GS.
Problem
Promptable 3D segmentation in radiance fields is slow and suffers from multi-granularity ambiguity; prior SAM-lifting methods either need iterative refinement or implicit feature fields that are costly at query time.
Approach
Attach a D=32 affinity feature to each 3D Gaussian and modulate its channels with a single linear+sigmoid scale gate conditioned on 3D physical scale; train with a scale-aware pixel-identity correspondence distillation loss from SAM masks, plus local KNN feature smoothing and feature-norm regularization.
Key results
- On NVOS, SAGA reaches 92.6 mIoU, versus 92.2 for SA3D-GS and 91.7 for OmniSeg3D.
- On 3D-OVS open-vocabulary segmentation, SAGA achieves 96.0 mean mIoU, above N2F2 (93.9) and LangSplat (93.4).
- SAGA inference takes 2-5 ms per prompt, compared with 50-100 ms for OmniSeg3D and 30-70 ms for GARField.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SAGA instead of SA3D-GS | 3D-GS promptable segmentation · NVOS | mIoU (%): +0.4 mIoU | Helped |
| Use SAGA instead of OmniSeg3D | 3D-GS promptable segmentation · SPIn-NeRF | mIoU (%): 94.3 -> 93.4 | Hurt |
| Use SAGA instead of N2F2 | open-vocabulary segmentation · 3D-OVS | mean mIoU: 93.9 -> 96.0 | Helped |
| Use SAGA instead of GARField | radiance-field promptable segmentation · Promptable segmentation inference | inference time: 30\textasciitilde 70 ms -> 2\textasciitilde 5 ms | Helped |
| Add local feature smoothing (ablation) | SAGA · 3D object segmentation | false positives | Helped |
| Add feature norm regularization (ablation) | SAGA · 3D object segmentation | feature alignment | Helped |
Surprising findings
- Fitting the scale gate reveals that across 47 scenes about 36% of gates are positive, with more gates closing at larger scales.
Limitations
- Can only segment objects that appear in the SAM-extracted masks, so small or unrecognized targets are hard to segment.
- Reflections in 3D-GS create outlier Gaussians that degrade SPIn-NeRF segmentation, leaving it slightly below OmniSeg3D.
- Open-vocabulary segmentation relies on CLIP over SAM crops, which misclassifies larger objects due to multi-granularity ambiguity and lacks context.
Experimental setup
Datasets: NVOS SPIn-NeRF 3D-OVS LLFF MIP-360 Tanks&Temple Replica
Benchmarks: NVOS SPIn-NeRF 3D-OVS
Metrics: mIoU mAcc inference time
Baselines: NVOS ISRF SA3D OmniSeg3D GauGroup SA3D-GS MVSeg SA-GS LERF 3D-OVS LangSplat N2F2
Builds on: 3D Gaussian Splatting GARField Segment Anything Model LERF
Closest papers by meaning
- GaussianCut: Interactive segmentation via graph cut for 3D Gaussian Splatting (NeurIPS 2024 · similarity 0.86)
- PanoGS: Gaussian-based Panoptic Segmentation for 3D Open Vocabulary Scene Understanding (CVPR 2025 · similarity 0.86)
- Efficient Decoupled Feature 3D Gaussian Splatting via Hierarchical Compression (CVPR 2025 · similarity 0.85)
- Rethinking End-to-End 2D to 3D Scene Segmentation in Gaussian Splatting (CVPR 2025 · similarity 0.85)
- Segment then Splat: Unified 3D Open-Vocabulary Segmentation via Gaussian Splatting (NeurIPS 2025 · similarity 0.85)
- Dr. Splat: Directly Referring 3D Gaussian Splatting via Direct Language Embedding Registration (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.