Research map / 3D vision / Gaussian splatting
FastLGS: Speeding Up Language Embedded Gaussians with Feature Grid Mapping
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FastLGS, which builds a semantic feature grid from SAM-mask CLIP features and maps them to low-dimensional features trained in 3D Gaussian Splatting for fast, accurate open-vocabulary 3D querying.
Problem
Open-vocabulary semantic querying in radiance fields struggles to jointly achieve high accuracy, efficiency (real-time, high resolution), and zero-shot ability; prior methods like LERF and LangSplat are slow or produce fuzzy/inconsistent relevancy.
Approach
Store multi-view CLIP features of SAM masks in a semantic feature grid, cross-view match masks (SIFT keypoints plus CLIP and color-distribution similarity) to assign shared grid indices, train only low-dimensional features per Gaussian, and restore pixel-aligned CLIP embeddings from the grid at query time instead of using an MLP decoder.
Key results
- On the 3D-OVS dataset FastLGS reaches 94.4% overall mIoU versus 92.7% for LangSplat and 86.8% for 3D-OVS.
- On the LERF dataset FastLGS reaches 91.7% overall localization accuracy versus 84.3% for LangSplat.
- Query time at 1440x1080 is 0.52 s per query on the sofa scene versus 2.14 s for LangSplat and 51.2 s for LERF, with a 200MB model size.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FastLGS instead of LangSplat | Open-vocabulary object retrieval · SPIn-NeRF | mIoU (%): 92.2 -> 93.1 | Helped |
| Use FastLGS instead of LangSplat | Open-vocabulary object retrieval · SPIn-NeRF | mTime (s): 1.43 -> 0.31 | Helped |
| Use FastLGS instead of LERF | Open-vocabulary object retrieval · SPIn-NeRF | mAcc (%): 99.5 -> 95.2 | Hurt |
| Use FastLGS instead of LangSplat | Language-guided localization · LERF dataset overall | localization accuracy: 84.3 -> 91.7 | Helped |
| Use FastLGS instead of LangSplat | Open-vocabulary segmentation · 3D-OVS | mIoU (%): 92.7 -> 94.4 | Helped |
| Add a semantic feature grid to 3D-GS (ablation) | SAM-based CLIP feature rendering · Querying feature ablation | mIoU (%): OOM -> 95.1 | Helped |
| Add a semantic feature grid to 3D-GS (ablation) | SAM-based CLIP feature rendering · Querying feature ablation | mTime (s): OOM -> 0.98 | Helped |
| Add keypoint correspondence (ablation) | Cross-view feature matching · Semantic feature grid ablation | mIoU (%): 75.2 -> 87.5 | Helped |
| Add keypoint correspondence (ablation) | Cross-view feature matching · Semantic feature grid ablation | mAcc (%): 83.2 -> 91.8 | Helped |
| Add color distribution features (ablation) | Cross-view feature matching with keypoint correspondence · Semantic feature grid ablation | mIoU (%): 87.5 -> 92.1 | Helped |
| Add color distribution features (ablation) | Cross-view feature matching with keypoint correspondence · Semantic feature grid ablation | mAcc (%): 91.8 -> 97.5 | Helped |
Surprising findings
- Naively adding high-dimensional CLIP features to 3DGS ran out of memory, motivating the feature grid design.
Limitations
- Struggles with queries for object parts (e.g., 'bicycle handlebars'), since it follows an object-centric paradigm.
- Relies on SAM masks and SIFT keypoint matching, so cross-view consistency depends on mask and matching quality.
Experimental setup
Datasets: SPIn-NeRF LERF 3D-OVS
Benchmarks: SPIn-NeRF LERF dataset 3D-OVS
Metrics: mIoU mAcc localization accuracy query time (s) model size
Baselines: LERF LangSplat LEGaussians MVSeg SA3D ODISE OV-Seg 3D-OVS SAM (2D)
Builds on: 3D Gaussian Splatting LERF LangSplat SAM SPIn-NeRF
Closest papers by meaning
- Feature 3DGS: Supercharging 3D Gaussian Splatting to Enable Distilled Feature Fields (CVPR 2024 · similarity 0.92)
- FHGS: Feature-Homogenized Gaussian Splatting (NeurIPS 2025 · similarity 0.90)
- SceneSplat++: A Large Dataset and Comprehensive Benchmark for Language Gaussian Splatting (NeurIPS 2025 · similarity 0.90)
- Tackling View-Dependent Semantics in 3D Language Gaussian Splatting (ICML 2025 · similarity 0.89)
- Rethinking Open-Vocabulary Segmentation of Radiance Fields in 3D Space (AAAI 2025 · similarity 0.89)
- Language Embedded 3D Gaussians for Open-Vocabulary Scene Understanding (CVPR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.