Research map / Detection and segmentation / 3D detection and driving
GREAT: Geometry-Intention Collaborative Inference for Open-Vocabulary 3D Object Affordance Grounding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GREAT, which fine-tunes InternVL with multi-head affordance chain-of-thought prompts to infer object geometry and interaction intentions, fuses them with point and image features for open-vocabulary 3D affordance grounding, and releases the PIADv2 dataset.
Problem
Open-vocabulary 3D object affordance grounding: localizing action-possibility regions on 3D objects for arbitrary, unseen affordances and objects, which prior methods fail to generalize to because they rely on a limited learned semantic space.
Approach
Use an MLLM with a multi-head chain-of-thought (object-geometry reasoning and affordance/interaction-intention brainstorming with analogical reasoning) on interaction images to produce knowledge features, then inject them into point features via a cross-modal adaptive fusion module and fuse with image features for point-wise affordance heatmap prediction.
Key results
- On PIADv2 Unseen Affordance, GREAT reaches 69.81 AUC and 12.05 aIOU versus 64.07 AUC and 8.37 aIOU for LASO.
- On PIADv2 Unseen Object, GREAT reaches 79.57 AUC and 20.16 aIOU, outperforming the best baseline (XMF, 74.61 AUC; FRCNN 18.08 aIOU).
- On the Seen partition, GREAT achieves 91.99 AUC and 38.03 aIOU, above LASO's 90.34 AUC and 34.88 aIOU.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GREAT instead of LASO | 3D affordance grounding · Seen | AUC: 90.34 -> 91.99 | Helped |
| Use GREAT instead of LASO | 3D affordance grounding · Unseen Object | AUC: 73.32 -> 79.57 | Helped |
| Use GREAT instead of LASO | 3D affordance grounding · Unseen Affordance | AUC: 64.07 -> 69.81 | Helped |
| Add affordance-head CoT (ablation) | GREAT · Unseen Affordance | AUC: 67.18 -> 69.81 | Helped |
| Add object-head CoT (ablation) | GREAT · Unseen Affordance | AUC: 64.69 -> 69.81 | Helped |
| Add CMAFM (ablation) | GREAT · Unseen Affordance | AUC: 63.00 -> 69.81 | Helped |
| Add MLLM fine-tuning (ablation) | GREAT · Unseen Affordance | AUC: 66.49 -> 69.81 | Helped |
Limitations
- High computational cost of multi-step MLLM reasoning, a bottleneck for large-scale or real-time use.
- Inference relies on running a fine-tuned MLLM per interaction image, which is expensive.
Experimental setup
Datasets: PIADv2 PIAD 3D-AffordanceNet AGD20k OpenImage 3DIR Objaverse
Benchmarks: PIADv2
Metrics: AUC aIOU SIM MAE
Baselines: Baseline (concatenation) FRCNN XMF IAG LASO
Builds on: InternVL PIAD PointNet++ LoRA
Closest papers by meaning
- Grounding 3D Object Affordance with Language Instructions, Visual Observations and Interactions (CVPR 2025 · similarity 0.93)
- Learning 2D Invariant Affordance Knowledge for 3D Affordance Grounding (AAAI 2025 · similarity 0.91)
- Probing and Bridging Geometry-Interaction Cues for Affordance Reasoning in Vision Foundation Models (CVPR 2026 · similarity 0.90)
- AffordBot: 3D Fine-grained Embodied Reasoning via Multimodal Large Language Models (NeurIPS 2025 · similarity 0.90)
- SeqAfford: Sequential 3D Affordance Reasoning via Multimodal Large Language Model (CVPR 2025 · similarity 0.90)
- 3D-AffordanceLLM: Harnessing Large Language Models for Open-Vocabulary Affordance Detection in 3D Worlds (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.