Research map / Vision-language and multimodal / Human and embodied modeling
DenseGrounding: Improving Dense Language-Vision Semantics for Ego-centric 3D Visual Grounding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DenseGrounding, which adds a Hierarchical Scene Semantic Enhancer for dense visual semantics and an LLM-based Language Semantic Enhancer that augments training descriptions using a scene database, improving ego-centric multi-view 3D visual grounding on EmbodiedScan.
Problem
Ego-centric 3D visual grounding suffers from loss of fine-grained visual semantics due to sparse point-cloud/image fusion (about 2% of points sampled) and limited, ambiguous textual descriptions in training data.
Approach
Enrich sparse visual features with scene-level global semantics via view-level aggregation, scene-level interaction with language, and broadcast back to multi-scale point features; and augment training descriptions with GPT-4o-mini grounded in a scene information database of object relations and locations, used only during training.
Key results
- On the full training set, DenseGrounding reaches 44.41% overall ACC@0.25 on the EmbodiedScan validation set, +5.81 over EmbodiedScan (38.60%).
- With the mini training set, DenseGrounding reaches 41.34% overall, +7.56 over the original EmbodiedScan (33.78%) and +5.57 over the enhanced baseline.
- Ablation: LSE alone raises mini-set overall accuracy from 35.77 to 39.25 and HSSE alone to 39.02; combined gives 41.34.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add LSE and HSSE | EmbodiedScan · EmbodiedScan full validation | Overall ACC25: +5.81 | Helped |
| Add LSE and HSSE | Enhanced EmbodiedScan · EmbodiedScan mini validation | Overall ACC25: +5.57% | Helped |
| Add LSE and HSSE | EmbodiedScan trained on 3RScan and Matterport · ScanNet zero-shot test | Overall accuracy: 9.81 -> 18.14 | Helped |
| Add SIDB object relationships to LLM text augmentation (ablation) | LLM augmentation · EmbodiedScan mini validation | Overall accuracy: 37.23 -> 38.08 | Helped |
| Add object locations to SIDB alongside relationships (ablation) | LLM augmentation · EmbodiedScan mini validation | Overall accuracy: 38.08 -> 38.82 | Helped |
| Add HSSE without LSE (ablation) | Enhanced EmbodiedScan · EmbodiedScan mini validation | Overall accuracy: 35.77 -> 39.02 | Helped |
| Add LSE without HSSE (ablation) | Enhanced EmbodiedScan · EmbodiedScan mini validation | Overall accuracy: 35.77 -> 39.25 | Helped |
| Add both LSE and HSSE (ablation) | Enhanced EmbodiedScan · EmbodiedScan mini validation | Overall accuracy: 35.77 -> 41.34 | Helped |
| Use 3 rather than 1 scene-attention layer (ablation) | HSSE · EmbodiedScan mini validation | Overall accuracy: 39.77 -> 41.34 | Helped |
| Use 3 rather than 2 scene-attention layers (ablation) | HSSE · EmbodiedScan mini validation | Overall accuracy: 40.11 -> 41.34 | Helped |
| Use 4 rather than 3 scene-attention layers (ablation) | HSSE · EmbodiedScan mini validation | Overall accuracy: 41.34 -> 40.49 | Hurt |
| Use 6 rather than 3 scene-attention layers (ablation) | HSSE · EmbodiedScan mini validation | Overall accuracy: 41.34 -> 40.06 | Hurt |
Surprising findings
- Training on 40% of data with the method surpasses the baseline trained on 80% of data, per the limited-data analysis.
Limitations
- Vague or ambiguous human descriptions at inference remain challenging since augmentation is training-only.
- Relies on the annotated scene database and detection-model pseudo-labels for location information in the LSE pipeline.
Experimental setup
Datasets: EmbodiedScan ScanNet 3RScan Matterport3D
Benchmarks: EmbodiedScan
Metrics: ACC@0.25 overall Easy/Hard ACC@0.25 Dep/Indep ACC@0.25
Baselines: ScanRefer BUTD-DETR L3Det EmbodiedScan ViewRefer
Builds on: EmbodiedScan ViewRefer DETR CLIP
Closest papers by meaning
- Connecting the Dots: Training-Free Visual Grounding via Agentic Reasoning (AAAI 2026 · similarity 0.91)
- G^3-LQ: Marrying Hyperbolic Alignment with Explicit Semantic-Geometric Modeling for 3D Visual Grounding (CVPR 2024 · similarity 0.91)
- Towards CLIP-driven Language-free 3D Visual Grounding via 2D-3D Relational Enhancement and Consistency (CVPR 2024 · similarity 0.91)
- Grounded 3D-Aware Spatial Vision-Language Modeling (CVPR 2026 · similarity 0.90)
- AugRefer: Advancing 3D Visual Grounding via Cross-Modal Augmentation and Spatial Relation-based Referring (AAAI 2025 · similarity 0.90)
- GroundingME: Exposing the Visual Grounding Gap in MLLMs through Multi-Dimensional Evaluation (CVPR 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.