Research map / 3D vision / Gaussian splatting
LangSplatV2: High-dimensional 3D Language Gaussian Splatting with 450+ FPS
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LangSplatV2, which replaces LangSplat's MLP decoder with a 3D sparse coefficient field over a global codebook plus sparse coefficient splatting, reaching ~476 FPS feature splatting and ~385 FPS open-vocabulary querying.
Problem
Real-time open-vocabulary 3D language querying with Gaussian splatting is bottlenecked by the MLP decoder that maps low-dimensional latent features back to high-dimensional CLIP features, and rendering high-dimensional features directly is slow.
Approach
Model each Gaussian's CLIP embedding as a sparse code (top-K of L=64 coefficients) over a global learned codebook of D-dimensional basis vectors, so rendering only needs to splat K-dimensional sparse coefficients, with a matrix multiply replacing the decoder.
Key results
- LangSplatV2 reaches 384.6 FPS for open-vocabulary 3D querying versus 8.2 FPS for LangSplat (47x) on LERF with one A100.
- LangSplatV2 achieves 84.1% localization accuracy and 59.9% segmentation IoU on LERF, versus LangSplat's 84.3% and 51.4%.
- On 3D-OVS LangSplatV2 reaches 94.6% mean IoU (LangSplat 93.4%); on Mip-NeRF360 it reaches 69.4% (LangSplat 57.3%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace LangSplat's latent decoder with a 3D sparse coefficient field and global codebook | LangSplatV2 · LERF query | query time: 122.1 -> 2.6 ms | Helped |
| Use LangSplatV2 instead of LangSplat | LERF 3D object localization · LERF | accuracy: 84.3 -> 84.1 | Mixed |
| Use LangSplatV2 instead of LangSplat | LERF 3D semantic segmentation · LERF | IoU: 51.4 -> 59.9 | Helped |
| Use LangSplatV2 instead of LangSplat | 3D-OVS semantic segmentation · 3D-OVS | IoU: 93.4 -> 94.6 | Helped |
| Use LangSplatV2 instead of LangSplat | Mip-NeRF360 semantic segmentation · Mip-NeRF360 | IoU: 57.3 -> 69.4 | Helped |
| Increase codebook size from L=32 to L=64 (ablation) | LangSplatV2 · LERF overall | accuracy: 72.8% to 84.1% | Helped |
| Increase codebook size from L=32 to L=64 (ablation) | LangSplatV2 · LERF overall | IoU: 53.9% to 59.9% | Helped |
| Increase codebook size from L=64 to L=128 (ablation) | LangSplatV2 · LERF overall | accuracy: 84.1 -> 84.1 | No clear effect |
| Increase codebook size from L=64 to L=128 (ablation) | LangSplatV2 · LERF overall | IoU: 59.9 -> 60.5 | Helped |
| Increase sparsity K from 2 to 4 (ablation) | LangSplatV2 · LERF overall | accuracy: 79.4 -> 84.1 | Helped |
| Increase sparsity K from 2 to 4 (ablation) | LangSplatV2 · LERF overall | IoU: 54.4 -> 59.9 | Helped |
| Increase sparsity K from 4 to 6 (ablation) | LangSplatV2 · LERF overall | accuracy: 84.1 -> 84.4 | Helped |
Surprising findings
- Training LangSplatV2 takes 3.0 h versus 1.0 h for LangSplat, a trade-off the authors accept for faster inference.
- On LERF object localization LangSplatV2 is slightly below LangSplat on the Kitchen scene (86.4% vs 95.5%) and its overall accuracy is marginally below LangSplat's (84.1% vs 84.3%).
Limitations
- Training cost is higher (3.0 h and 21.2 GB vs 1.0 h and 6.2 GB for LangSplat) because high-dimensional semantic fields are built during training.
- Inherits biases from the CLIP model.
- Codebook size L and sparsity K are hyperparameters; performance saturates at L=64 and K=4 on LERF, and larger K slows rendering.
Experimental setup
Datasets: LERF 3D-OVS Mip-NeRF360
Benchmarks: LERF 3D-OVS Mip-NeRF360
Metrics: localization accuracy mean IoU query time (ms) FPS GPU memory
Baselines: LangSplat LEGaussians GOI GAGS GS-Grouping LERF 3D-OVS FFD
Builds on: LangSplat 3D Gaussian Splatting LERF SAM
Closest papers by meaning
- LangSplat: 3D Language Gaussian Splatting (CVPR 2024 · similarity 0.93)
- SceneSplat++: A Large Dataset and Comprehensive Benchmark for Language Gaussian Splatting (NeurIPS 2025 · similarity 0.90)
- 4D LangSplat: 4D Language Gaussian Splatting via Multimodal Large Language Models (CVPR 2025 · similarity 0.90)
- SOVGaussian: Sparse-View 3D Gaussian Splatting for Open-Vocabulary Scene Understanding (AAAI 2025 · similarity 0.89)
- Dr. Splat: Directly Referring 3D Gaussian Splatting via Direct Language Embedding Registration (CVPR 2025 · similarity 0.88)
- Language Embedded 3D Gaussians for Open-Vocabulary Scene Understanding (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.