Research map / Detection and segmentation / 3D detection and driving
Intent3D: 3D Object Detection in RGB-D Scans Based on Human Intention
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces 3D intention grounding (detecting 3D objects in RGB-D/point-cloud ScanNet scenes from free-form human intention text), releases the Intent3D dataset with 44,990 GPT-4-generated intention texts, and proposes IntentNet with verb-object alignment, candidate box matching, and cascaded adaptive losses.
Problem
Detecting 3D objects in indoor scenes directly from human intention statements (e.g., "I want something to support my back") rather than referential descriptions, which requires the AI to reason about which objects fulfill the intention.
Approach
Reformulate 3D grounding around intentions by aligning verbs then verb-conditioned objects, add candidate box matching over detector proposals, and weight a chain of losses with cascaded sigmoid-based adaptive factors.
Key results
- IntentNet achieves 58.34% Top1-Acc@0.25 and 40.83% Top1-Acc@0.5 on the Intent3D val set, the best among baselines (BUTD-DETR 47.12 / 24.56; 3D-VisTA 42.76 / 30.37).
- On the test set IntentNet reaches 58.92% Top1-Acc@0.25, 42.28% Top1-Acc@0.5, 44.01 AP@0.25 and 27.60 AP@0.5.
- Ablation on val shows removing Verb alignment drops Top1-Acc@0.25 from 58.34 to 53.09 and removing Cascaded Adaptive Learning drops it to 57.39.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use IntentNet instead of BUTD-DETR | IntentNet · Intent3D val | Top1-Acc@0.25: 47.12 -> 58.34 | Helped |
| Use IntentNet instead of 3D-VisTA | IntentNet · Intent3D val | AP@0.5: 19.93 -> 25.36 | Helped |
| Use IntentNet instead of BUTD-DETR | IntentNet · Intent3D test | Top1-Acc@0.25: 47.86 -> 58.92 | Helped |
| Use IntentNet instead of 3D-VisTA | IntentNet · Intent3D test | AP@0.5: 22.00 -> 27.60 | Helped |
| Use IntentNet instead of GPT-4 two-stage selection | IntentNet · Intent3D val | ap@0.25: 15.10 -> 41.90 | Helped |
| Add verb-token alignment (ablation) | IntentNet · Intent3D val | Top1-Acc@0.25: 53.09 -> 58.34 | Helped |
| Add verb-conditioned object-token alignment (ablation) | IntentNet · Intent3D val | Top1-Acc@0.25: 57.87 -> 58.34 | Helped |
| Add Candidate Box Matching (ablation) | IntentNet · Intent3D val | Top1-Acc@0.25: 56.25 -> 58.34 | Helped |
| Add Cascaded Adaptive Learning (ablation) | IntentNet · Intent3D val | Top1-Acc@0.25: 57.39 -> 58.34 | Helped |
| Add verb-token alignment (ablation) | IntentNet · Intent3D val | Top1-Acc@0.5: 34.62 -> 40.83 | Helped |
| Add verb-conditioned object-token alignment (ablation) | IntentNet · Intent3D val | Top1-Acc@0.5: 39.42 -> 40.83 | Helped |
| Add Candidate Box Matching (ablation) | IntentNet · Intent3D val | Top1-Acc@0.5: 38.37 -> 40.83 | Helped |
Surprising findings
- Chat-3D-v2 zero-shot performs near chance (5.86% Top1-Acc@0.25), and fine-tuned Chat-3D-v2 has very low AP (3.23 AP@0.25) despite decent Top1 accuracy, attributed to false positives.
- EDA, a stronger 3D-VG model, performs worse than BUTD-DETR on Intent3D, attributed to aligning with auxiliary nouns that are not the target.
Limitations
- Performance drops on intentions involving multiple actions, where secondary actions distract the model.
- Dataset relies on GPT-4-generated intention texts, introducing possible subjectivity and reliance on a single LLM generator.
- Ambiguity in one-to-many intention-object mapping; the dataset enforces unambiguous mapping per scene.
Experimental setup
Datasets: ScanNet Intent3D
Benchmarks: Intent3D
Metrics: Top1-Acc@0.25 Top1-Acc@0.5 AP@0.25 AP@0.5
Baselines: BUTD-DETR EDA 3D-VisTA Chat-3D-v2 GPT-4 two-stage
Builds on: ScanRefer BUTD-DETR EDA 3D-VisTA Group-Free
Closest papers by meaning
- GREAT: Geometry-Intention Collaborative Inference for Open-Vocabulary 3D Object Affordance Grounding (CVPR 2025 · similarity 0.86)
- ScanERU: Interactive 3D Visual Grounding Based on Embodied Reference Understanding (AAAI 2024 · similarity 0.85)
- Multi-Object 3D Grounding with Dynamic Modules and Language-Informed Spatial Attention (NeurIPS 2024 · similarity 0.84)
- Grounding 3D Object Affordance with Language Instructions, Visual Observations and Interactions (CVPR 2025 · similarity 0.84)
- Grounded 3D-Aware Spatial Vision-Language Modeling (CVPR 2026 · similarity 0.84)
- From Objects to Anywhere: A Holistic Benchmark for Multi-level Visual Grounding in 3D Scenes (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.