Research map / 3D vision / NeRF and radiance fields
GARField: Group Anything with Radiance Fields
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Distills SAM masks into a scale-conditioned 3D affinity field on a NeRF (Nerfacto), then recursively clusters it with HDBSCAN to get a view-consistent hierarchy of 3D groups.
Problem
Grouping scene content into semantically meaningful hierarchies in 3D is ambiguous because the same point can belong to groups at several granularities, and 2D SAM masks are inconsistent across views.
Approach
Condition a 3D affinity feature field on physical scale, so that conflicting 2D masks from different views and granularities coexist at different scales; train with a scale-conditioned contrastive loss plus densified scale supervision and a containment loss, then cluster recursively at descending scales.
Key results
- On 3D completeness, GARField improves mIOU over SAM, e.g. keyboard fine level 65.3 to 88.8 and bouquet fine level 17.4 to 76.0.
- On hierarchical grouping recall, GARField reaches 85.6 (ramen), 86.6 (teatime), 77.9 (keyboard), 76.4 (bouquet), 76.6 (living room) mIOU vs SAM's 74.9, 64.9, 23.2, 34.4, 59.6.
- Removing scale conditioning or dense supervision reduces hierarchical recall, showing both components are necessary.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GARField instead of SAM masks | 3D completeness evaluation · teatime, bouquet, keyboard, ramen, and living room (fine, medium, and coarse levels) | mIOU: 81.6 -> 92.7; 97.3 -> 97.9; 17.4 -> 76.0; 73.5 -> 81.6; 76.1 -> 85.4; 65.3 -> 88.8; 73.6 -> 98.4; 53.3 -> 79.2; 74.7 -> | Helped |
| Use GARField instead of SAM automatic masks | hierarchical grouping recall · ramen, teatime, keyboard, bouquet, and living room | mIOU: 74.9 vs 74.9; 64.9 vs 64.9; 23.2 vs 23.2; 34.4 vs 34.4; 59.6 vs 59.6 | Mixed |
| Add scale-conditioning (ablation) | GARField · ramen, teatime, keyboard, bouquet, and living room | mIOU: 64.1 -> 74.9; 67.7 -> 64.9; 57.6 -> 23.2; 49.8 -> 34.4; 49.7 -> 59.6 | Mixed |
| Add densified supervision (ablation) | GARField · ramen, teatime, keyboard, bouquet, and living room | mIOU: 85.6 vs 74.9; 86.6 vs 64.9; 77.9 vs 23.2; 76.4 vs 34.4; 76.6 vs 59.6 | Hurt |
Limitations
- Output is bounded by the quality of the 2D mask generator; groups missing from SAM masks cannot emerge in 3D.
- Scale alone cannot resolve all grouping ambiguity, e.g. an object inside a container shares scale with the container.
- Large background groups may fail due to scale clamping at 2x camera extent.
Experimental setup
Datasets: Nerfstudio dataset LERF dataset SAM automatic mask outputs
Benchmarks: 3D completeness annotations on 5 scenes hierarchical grouping recall on 5 scenes
Metrics: mIOU
Baselines: SAM GARField (-scale) GARField (-dense)
Builds on: Segment Anything Nerfacto LERF Contrastive Lift HDBSCAN
Closest papers by meaning
- Feature 3DGS: Supercharging 3D Gaussian Splatting to Enable Distilled Feature Fields (CVPR 2024 · similarity 0.84)
- Towards Learning Group-Equivariant Features for Domain Adaptive 3D Detection (NeurIPS 2024 · similarity 0.83)
- Rethinking Open-Vocabulary Segmentation of Radiance Fields in 3D Space (AAAI 2025 · similarity 0.83)
- GSNeRF: Generalizable Semantic Neural Radiance Fields with Enhanced 3D Scene Understanding (CVPR 2024 · similarity 0.83)
- SAI3D: Segment Any Instance in 3D Scenes (CVPR 2024 · similarity 0.83)
- MS-GS: Multi-Appearance Sparse-View 3D Gaussian Splatting in the Wild (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.