Research map / 3D vision / Gaussian splatting
Gaussian Splatting Feature Fields for (Privacy-Preserving) Visual Localization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Gaussian Splatting Feature Fields (GSFFs), a self-supervised 3DGS-plus-triplane feature field jointly trained with a 2D encoder, enabling accurate feature-based and privacy-preserving segmentation-based visual localization via pose refinement.
Problem
Visual localization that is both accurate and privacy-preserving: feature descriptors can leak image details, while existing privacy-preserving methods (segmentation-based) are less accurate and lack multi-view consistency.
Approach
Combine an explicit 3D Gaussian Opacity Fields model with a scale-aware triplane feature field, trained jointly with a 2D encoder via contrastive losses, multi-view consistency, and spectral-clustering prototypes; convert features to discrete per-Gaussian labels for privacy-preserving pose refinement by differentiable rasterization.
Key results
- GSFFs-PR Feature is best on 4 of 7 7Scenes scenes (SfM pseudo GT) and reaches 0.4 cm / 0.19 deg on Chess, outperforming rendering-based baselines.
- On Cambridge Landmarks, GSFFs-PR Privacy achieves King's College 24 cm / 0.39 deg vs SegLoc 24 cm / 0.26 deg, and is much more accurate than other privacy-preserving methods like GoMatch and DGC-GNN.
- On Indoor6, GSFFs-PR Privacy with 84 classes outperforms SegLoc in average recall.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use GSFFs-PR Feature pose refinement | 7Scenes · 7Scenes (SfM pseudo GT), Chess | median position error (cm): 0.5 -> 0.4 | Helped |
| use GSFFs-PR Feature pose refinement | 7Scenes · 7Scenes (SfM pseudo GT), Stairs | median position error (cm): 1.4 -> 25.1 | Hurt |
| use GSFFs-PR Privacy pose refinement | 7Scenes privacy-preserving localization · 7Scenes (DSLAM pseudo GT), Stairs | median position error (cm): 43.7 -> 28.3 | Helped |
| add a fine level to coarse-to-fine refinement (ablation) | GSFFs-PR Feature · King’s College | median position error (cm): 37.5 -> 17.9 | Helped |
| add a coarse level to fine-level refinement (ablation) | GSFFs-PR Privacy · King’s College | median position error (cm): 28.0 -> 24.3 | Helped |
| use spectral clustering instead of k-means (ablation) | GSFFs-PR Feature · King’s College | median position error (cm): 25.2 -> 17.9 | Helped |
| add segmentation optimization (ablation) | GSFFs-PR Feature · King’s College | median position error (cm): 23.5 -> 17.9 | Helped |
| add multi-view regularization (ablation) | GSFFs-PR Feature · King’s College | median position error (cm): 20.0 -> 17.9 | Helped |
| use scale-aware feature encoding instead of point-wise Gaussian-center projection (ablation) | GSFFs-PR Feature · King’s College | median position error (cm): 23.5 -> 17.9 | Helped |
| add a fine level to coarse-only refinement (ablation) | GSFFs-PR Feature · Old Hospital | median position error (cm): 670 -> 21.4 | Helped |
| use spectral clustering instead of k-means (ablation) | GSFFs-PR Privacy · King’s College | median position error (cm): 37.2 -> 24.3 | Helped |
| increase segmentation classes from 34 to 84 | GSFFs-PR Privacy · Indoor6 | recall at 5cm/5° | Mixed |
Surprising findings
- Sparse-only pose refinement using only Gaussian centers fails to converge, indicating dense rendering is needed.
- GSFFs-PR Feature fails on the Stairs scene of 7Scenes (25.1 cm) due to the textureless layout hurting the 3DGS initial structure.
Limitations
- Stairs scene fails due to poor 3D structure from textureless, flat layout.
- Requires training one model per scene (50K iterations) and relies on a global descriptor for initialization.
Experimental setup
Datasets: 7Scenes Cambridge Landmarks Indoor6 12Scenes
Benchmarks: 7Scenes Cambridge Landmarks Indoor6
Metrics: median position error (cm) median rotation error (deg) recall at 5cm/5deg
Baselines: HLoc DSAC* ACE ACE + GSLoc NeFeS MCLoc SSL-Nif NeRFMatch GSplatLoc GSLoc SegLoc GoMatch
Builds on: 3D Gaussian Splatting Gaussian Opacity Fields SegLoc NeRF feature field localization (SSL-Nif) triplane representation
Closest papers by meaning
- GS-SLAM: Dense Visual SLAM with 3D Gaussian Splatting (CVPR 2024 · similarity 0.87)
- VA-GS: Enhancing the Geometric Representation of Gaussian Splatting via View Alignment (NeurIPS 2025 · similarity 0.87)
- GPVK-VL: Geometry-Preserving Virtual Keyframes for Visual Localization under Large Viewpoint Changes (CVPR 2025 · similarity 0.87)
- GS-CPR: Efficient Camera Pose Refinement via 3D Gaussian Splatting (ICLR 2025 · similarity 0.87)
- GaussianUDF: Inferring Unsigned Distance Functions through 3D Gaussian Splatting (CVPR 2025 · similarity 0.87)
- FreeSplat: Generalizable 3D Gaussian Splatting Towards Free View Synthesis of Indoor Scenes (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.