Research map / Point clouds / Point cloud classification
Learning 2D Invariant Affordance Knowledge for 3D Affordance Grounding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MIFAG, which grounds 3D object affordance regions by iteratively extracting invariant affordance knowledge from multiple human-object interaction images and fusing it into point cloud features, evaluated on a new MIPA benchmark.
Problem
3D object affordance grounding from a single reference human-object interaction image generalizes poorly because the appearance and geometry of reference images and 3D objects are inconsistent.
Approach
Learn invariant affordance knowledge shared across multiple same-category interaction images via an iterative dual-branch cross-attention module (IAM) that builds an affordance dictionary, then fuse the dictionary into point cloud features with dictionary-based cross-attention and self-weighted attention (ADM).
Key results
- On MIPA seen split, MIFAG reaches AUC 85.10, aIOU 20.50, SIM 0.568, MAE 0.091, beating LASO and IAGNet.
- On MIPA unseen split, MIFAG reaches AUC 71.13 and aIOU 5.23, the best among compared methods.
- Ablation shows IAM alone raises seen AUC from 82.97 to 84.80, and combining IAM and ADM gives 85.10.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MIFAG instead of the strongest reported baseline | MIPA Seen · MIPA Seen | AUC: 83.13 -> 85.10 | Helped |
| Use MIFAG instead of the strongest reported baseline | MIPA Seen · MIPA Seen | aIOU: 17.92 -> 20.50 | Helped |
| Use MIFAG instead of the strongest reported baseline | MIPA Unseen · MIPA Unseen | AUC: 69.91 -> 71.13 | Helped |
| Use MIFAG instead of the strongest reported baseline | MIPA Unseen · MIPA Unseen | aIOU: 5.18 -> 5.23 | Helped |
| Add IAM (ablation) | MIFAG on MIPA Seen · MIPA Seen | AUC: 82.97 -> 84.80 | Helped |
| Add ADM (ablation) | MIFAG on MIPA Seen · MIPA Seen | AUC: 82.97 -> 84.57 | Helped |
| Add IAM (ablation) | MIFAG on MIPA Unseen · MIPA Unseen | AUC: 65.14 -> 70.34 | Helped |
| Add ADM (ablation) | MIFAG on MIPA Unseen · MIPA Unseen | AUC: 65.14 -> 69.91 | Helped |
| Increase image input from 1 to 5 (ablation) | MIFAG · MIPA Seen | AUC: 83.33 -> 85.47 | Helped |
| Increase image input from 1 to 5 (ablation) | MIFAG · MIPA Unseen | aIOU: 4.72 -> 4.49 | Hurt |
| Set iterative layer count to 4 for seen data and 6 for unseen data (ablation) | MIFAG · MIPA Seen and Unseen | AUC: 85.10; 71.13 | Mixed |
Surprising findings
- Adding more reference images does not monotonically help; unseen aIOU drops from 4.72 (1 image) to 4.49 at 5 images, with other settings fluctuating.
- Simple 3D segmentation and tracking baselines (PMF, XMF, MBDF) perform substantially worse than affordance-specific methods.
Limitations
- 3D objects in the MIPA dataset are relatively simple compared to real-world objects.
- The framework focuses on visual affordance understanding and does not model manipulation details such as manipulator size or action direction.
Experimental setup
Datasets: 3D AffordanceNet IAGNet (PIAD) HICO AGD20K
Benchmarks: MIPA
Metrics: AUC aIOU SIM MAE
Baselines: PMF MBDF FRCNN ILN PFusion XMF IAGNet LASO
Builds on: 3D AffordanceNet IAGNet PointNet++ ResNet-18
Closest papers by meaning
- Grounding 3D Object Affordance with Language Instructions, Visual Observations and Interactions (CVPR 2025 · similarity 0.92)
- GREAT: Geometry-Intention Collaborative Inference for Open-Vocabulary 3D Object Affordance Grounding (CVPR 2025 · similarity 0.91)
- GEAL: Generalizable 3D Affordance Learning with Cross-Modal Consistency (CVPR 2025 · similarity 0.90)
- Weakly Supervised Multimodal Affordance Grounding for Egocentric Images (AAAI 2024 · similarity 0.90)
- Weakly-Supervised Affordance Grounding Guided by Part-Level Semantic Priors (ICLR 2025 · similarity 0.89)
- IAAO: Interactive Affordance Learning for Articulated Objects in 3D Environments (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.