Research map / Vision-language and multimodal / Human and embodied modeling
ReasonGrounder: LVLM-Guided Hierarchical Feature Splatting for Open-Vocabulary 3D Visual Grounding and Reasoning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ReasonGrounder, an LVLM-guided hierarchical feature Gaussian splatting method for open-vocabulary 3D grounding and amodal reasoning, and releases the ReasoningGD synthetic dataset.
Problem
Open-vocabulary 3D visual grounding under implicit language instructions and occlusion, where existing methods rely on 3D annotations, mask proposals, and explicit prompts.
Approach
Builds a scale-hierarchical feature Gaussian field (language and instance features conditioned on physical mask scale from SAM masks and depth), uses LLaVA to infer the target object from an implicit query on a top-down view, selects a reference view via CLIP, and renders the matching Gaussian group for localization and amodal perception in novel views.
Key results
- On LERF, overall localization accuracy 86.7% vs. LangSplat 84.3%; mean IoU 55.1% vs. 51.4%.
- On 3D-OVS, mean IoU 94.7% vs. LangSplat 93.4%.
- On challenge scenes (Table 5), overall mean IoU 78.5% vs. LangSplat 58.9%; amodal perception on ReasoningGD achieves 90.4% mean IoU.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ReasonGrounder instead of LangSplat | open-vocabulary grounding on LERF · LERF | localization accuracy (%): 84.3 -> 86.7 | Helped |
| Use ReasonGrounder instead of LangSplat | open-vocabulary grounding · LERF | mean IoU (%): 51.4 -> 55.1 | Helped |
| Use ReasonGrounder instead of LangSplat | open-vocabulary grounding · 3D-OVS | mean IoU (%): 93.4 -> 94.7 | Helped |
| Use ReasonGrounder instead of LangSplat | grounding in challenge scenes · Challenge scenes | mean IoU (%): 58.9 -> 78.5 | Helped |
| Replace NeRF with 3DGS (ablation) | O-3DVG · Figurines | IoU (%): 47.2 -> 49.8 | Helped |
| Replace NeRF with 3DGS (ablation) | O-3DVG · Figurines | seconds per view: 0.924 -> 0.025 | Helped |
| Add LVLM for implicit instruction reasoning (ablation) | 3DGS O-3DVG · Figurines | IoU (%): 49.8 -> 49.3 | Hurt |
| Add scale-hierarchical feature Gaussian grouping (ablation) | 3DGS I-3DVG · Figurines | IoU (%): 48.9 -> 53.4 | Helped |
| Replace NeRF with 3DGS (ablation) | O-3DVG · ReasoningGD 001 | IoU (%): 85.4 -> 87.8 | Helped |
| Add LVLM for implicit instruction reasoning (ablation) | 3DGS O-3DVG · ReasoningGD 001 | IoU (%): 87.8 -> 87.7 | Hurt |
| Add scale-hierarchical feature Gaussian grouping (ablation) | 3DGS I-3DVG · ReasoningGD 001 | IoU (%): 86.9 -> 91.6 | Helped |
Limitations
- No direct open-vocabulary 3D reasoning baseline exists, so comparisons are against grounding-only methods.
- Gains over LangSplat on 3D-OVS and LERF are modest (e.g., 94.7 vs 93.4 mIoU).
- Relies on per-scene optimization and a 2D LVLM applied to a top-down view.
Experimental setup
Datasets: LERF 3D-OVS ReasoningGD
Benchmarks: LERF 3D-OVS ReasoningGD
Metrics: localization accuracy mean IoU
Baselines: LSeg LERF LangSplat ODISE OV-Seg FFD 3D-OVS
Builds on: 3D Gaussian Splatting LangSplat GARField LERF SAM
Closest papers by meaning
- Reasoning in Space via Grounding in the World (ICLR 2026 · similarity 0.91)
- S$^2$-MLLM: Boosting Spatial Reasoning Capability of MLLMs for 3D Visual Grounding with Structural Guidance (CVPR 2026 · similarity 0.91)
- DenseGrounding: Improving Dense Language-Vision Semantics for Ego-centric 3D Visual Grounding (ICLR 2025 · similarity 0.90)
- Grounded 3D-Aware Spatial Vision-Language Modeling (CVPR 2026 · similarity 0.90)
- From Objects to Anywhere: A Holistic Benchmark for Multi-level Visual Grounding in 3D Scenes (NeurIPS 2025 · similarity 0.90)
- G^3-LQ: Marrying Hyperbolic Alignment with Explicit Semantic-Geometric Modeling for 3D Visual Grounding (CVPR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.