Research map / Vision-language and multimodal / Human and embodied modeling
3EED: Ground Everything Everywhere in 3D
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces 3EED, a large-scale multi-platform (vehicle, drone, quadruped) multi-modal outdoor 3D visual grounding benchmark with VLM-generated and human-verified referring expressions, plus a scale-adaptive PointNet++ baseline.
Problem
Existing 3D visual grounding benchmarks are indoor-focused, single-platform, and small-scale, with limited multi-modal (LiDAR plus RGB) supervision, which limits generalization across embodied platforms.
Approach
A benchmark spanning vehicle, drone and quadruped platforms with synchronized LiDAR and RGB, a hybrid detector/tracker/human 3D box pipeline, and VLM-prompted referring expressions with human verification, together with a gravity-aligned cross-platform alignment, multi-scale sampling and scale-aware fusion baseline.
Key results
- Union-trained model reaches Acc@25 of 80.86 (Vehicle), 53.45 (Drone), 53.31 (Quadruped) versus best baseline EDA at 65.50, 46.00, 44.00.
- Training on Drone and testing zero-shot on Vehicle gives +13.93 Acc@25 over the strongest baseline.
- Multi-object grounding Acc@25 of 32.32 vs EDA 26.91 on Vehicle platform.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use the proposed scale-adaptive, platform-aware model instead of EDA | 3EED Vehicle-trained grounding · Vehicle test | Acc@25: 53.54 -> 78.37 | Helped |
| use the proposed scale-adaptive, platform-aware model instead of WildRefer | 3EED Vehicle-trained grounding · Drone test | Acc@25: 3.52 -> 18.16 | Helped |
| use the proposed scale-adaptive, platform-aware model instead of EDA | 3EED Vehicle-trained grounding · Quadruped test | Acc@25: 11.40 -> 36.04 | Helped |
| train on the union of platforms rather than Vehicle alone | Ours · 3EED Drone test | Acc@25: 18.16 -> 53.45 | Helped |
| train on the union of platforms rather than Vehicle alone | Ours · 3EED Quadruped test | Acc@25: 36.04 -> 53.31 | Helped |
| add cross-platform alignment (CPA) (ablation) | Ours · 3EED Vehicle test | Acc@25: 71.76 to 80.86 (+9.10) | Helped |
| add cross-platform alignment (CPA) (ablation) | Ours · 3EED Drone test | Acc@25: 51.84 -> 53.45 | Helped |
| add multi-scale sampling (MSS) (ablation) | Ours · 3EED Vehicle test | Acc@25: +5.21 | Helped |
| add multi-scale sampling (MSS) (ablation) | Ours · 3EED Drone test | Acc@25: 46.85 -> 53.45 | Helped |
| add multi-scale sampling (MSS) (ablation) | Ours · 3EED Quadruped test | Acc@25: 51.40 -> 53.31 | Helped |
| add scale-aware fusion (SAF) (ablation) | Ours · 3EED Vehicle test | Acc@25: 80.38 -> 80.86 | Helped |
| add scale-aware fusion (SAF) (ablation) | Ours · 3EED Drone test | Acc@25: 52.25 -> 53.45 | Helped |
Surprising findings
- Drone-trained and Quadruped-trained models show very low Acc@50 on drone and vehicle targets, indicating strong cross-platform gaps persist even with the proposed baseline.
- Drone scenes are sparsest (about 102 LiDAR points per object) yet contain the most objects per scene (8.05), making aerial grounding the hardest.
Limitations
- Drone and quadruped boxes are generated by a semi-automatic pipeline and human refinement, which may introduce pseudo-label noise.
- Cross-platform zero-shot performance remains low, especially for drone and Acc@50 metrics.
- Only two classes (Vehicle and Pedestrian) are focused on for grounding.
Experimental setup
Datasets: Waymo Open Dataset M3ED nuScenes Lyft
Benchmarks: 3EED
Metrics: Acc@25 Acc@50 mIoU
Baselines: BUTD-DETR EDA WildRefer
Builds on: EDA BUTD-DETR PointNet++ Waymo Open Dataset M3ED
Closest papers by meaning
- From Objects to Anywhere: A Holistic Benchmark for Multi-level Visual Grounding in 3D Scenes (NeurIPS 2025 · similarity 0.91)
- DenseGrounding: Improving Dense Language-Vision Semantics for Ego-centric 3D Visual Grounding (ICLR 2025 · similarity 0.89)
- CoT3DRef: Chain-of-Thoughts Data-Efficient 3D Visual Grounding (ICLR 2024 · similarity 0.89)
- Grounded 3D-Aware Spatial Vision-Language Modeling (CVPR 2026 · similarity 0.89)
- G^3-LQ: Marrying Hyperbolic Alignment with Explicit Semantic-Geometric Modeling for 3D Visual Grounding (CVPR 2024 · similarity 0.89)
- SeeGround: See and Ground for Zero-Shot Open-Vocabulary 3D Visual Grounding (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.