Research map / 3D vision / Gaussian splatting
Feature 3DGS: Supercharging 3D Gaussian Splatting to Enable Distilled Feature Fields
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Feature 3DGS distills arbitrary-dimension 2D foundation model features (LSeg, SAM) into 3D Gaussian Splatting via a parallel N-dimensional rasterizer and a 1x1 conv upsampling speed-up module, enabling semantic segmentation, language-guided editing, and promptable segmentation from novel views.
Problem
NeRF-based feature field distillation is slow to train and render, and implicit feature fields suffer from continuity artifacts and capacity trade-offs with radiance quality.
Approach
Attach a learnable low-dimensional semantic feature to each 3D Gaussian, rasterize it jointly with RGB through the same tile-based rasterizer, and use a lightweight 1x1 convolutional decoder to upsample to the teacher's feature dimension, avoiding the cost of rendering high-dimensional features directly.
Key results
- On Replica, Feature 3DGS (w/ speed-up) reaches 37.012 PSNR vs 36.133 for base 3DGS, and mIoU 0.782 vs 0.636 for NeRF-DFF with 14.55 FPS vs 5.38 FPS.
- Up to 2.7x faster feature distillation and rendering than NeRF-based distillation, per the paper's claims.
- SAM decoder-only rendering of novel-view features is up to 1.7x faster than encoder+decoder on rendered images with comparable mask quality.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| distill semantic features into 3D Gaussian Splatting rather than use Base 3DGS | Feature 3DGS · Replica | PSNR: 36.133 (±0.06) -> 36.915 (±0.05) | Helped |
| distill semantic features into 3D Gaussian Splatting rather than use Base 3DGS | Feature 3DGS · Replica | LPIPS: 0.033 (±1.2e-3) -> 0.024 (±1.1e-3) | Helped |
| use Feature 3DGS instead of NeRF-DFF | Feature 3DGS · Replica semantic segmentation | mIoU: 0.636 -> 0.787 | Helped |
| use Feature 3DGS instead of NeRF-DFF | Feature 3DGS · Replica semantic segmentation | FPS: 5.38 -> 6.84 | Helped |
| add the convolutional speed-up module (ablation) | Feature 3DGS · Replica | PSNR: 36.915 (±0.05) -> 37.012 (±0.07) | Helped |
| add the convolutional speed-up module (ablation) | Feature 3DGS · Replica semantic segmentation | mIoU: 0.787 -> 0.782 | Hurt |
| add the convolutional speed-up module (ablation) | Feature 3DGS · Replica semantic segmentation | FPS: 6.84 -> 14.55 | Helped |
| add the convolutional speed-up module (ablation) | Feature 3DGS · Replica semantic segmentation | accuracy: 0.943 -> 0.943 | No clear effect |
Surprising findings
- Joint equal-weighted feature and RGB optimization avoids the NeRF-style trade-off that requires a small feature loss weight, and even improves radiance quality over base 3DGS.
Limitations
- Student feature has limited access to ground-truth teacher features, bounding overall performance
- Teacher network imperfections propagate to the distilled feature field
- Adapted 3DGS pipeline produces noise-inducing floaters
Experimental setup
Datasets: Replica LLFF
Benchmarks: Replica LLFF
Metrics: PSNR SSIM LPIPS mIoU accuracy FPS
Baselines: NeRF-DFF Base 3DGS
Builds on: 3D Gaussian Splatting Distilled Feature Fields Segment Anything LSeg
Closest papers by meaning
- Compact 3D Gaussian Representation for Radiance Field (CVPR 2024 · similarity 0.93)
- FHGS: Feature-Homogenized Gaussian Splatting (NeurIPS 2025 · similarity 0.93)
- FastLGS: Speeding Up Language Embedded Gaussians with Feature Grid Mapping (AAAI 2025 · similarity 0.92)
- 3D Convex Splatting: Radiance Field Rendering with 3D Smooth Convexes (CVPR 2025 · similarity 0.91)
- FewViewGS: Gaussian Splatting with Few View Matching and Multi-stage Training (NeurIPS 2024 · similarity 0.91)
- Gaussian Herding across Pens: An Optimal Transport Perspective on Global Gaussian Reduction for 3DGS (NeurIPS 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.