Research map / Point clouds / Point cloud classification
GEAL: Generalizable 3D Affordance Learning with Cross-Modal Consistency
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GEAL, a dual-branch 3D affordance learner that renders point clouds into 2D via Gaussian splatting, uses DINOv2 features, and aligns 2D and 3D features to improve generalization and robustness, and introduces PIAD-C and LASO-C corruption benchmarks.
Problem
3D affordance learning from semantic cues (text/image) generalizes poorly to unseen objects and is fragile to real-world point cloud corruptions, due to limited annotations and geometry-focused 3D backbones.
Approach
Use 3D Gaussian splatting to render sparse point clouds into 2D images so a frozen pre-trained 2D foundation model (DINOv2) can supply semantics to the 3D branch, with granularity-adaptive fusion and a 2D-3D consistency loss transferring knowledge to the 3D PointNet++ branch used at inference.
Key results
- On PIAD Seen, GEAL reaches 22.5 aIoU and 0.600 SIM versus 20.5 aIoU for IAGNet; on Unseen it reaches 8.7 aIoU versus 8.0 for IAGNet and LASO.
- On LASO Unseen (reproduced split), GEAL achieves 16.7 aIoU vs 14.6 for LASO (original report).
- GEAL outperforms LASO on all seven corruption types on PIAD-C and LASO-C Seen partition (e.g., Drop-L aIoU 12.4 vs 10.6 on PIAD-C).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use GEAL instead of IAGNet | PIAD Seen · PIAD Seen | aIoU: 20.5 -> 22.5 | Helped |
| use GEAL instead of LASO | PIAD Unseen · PIAD Unseen | AUC: 69.2 -> 72.5 | Helped |
| use GEAL instead of LASO | LASO Seen · LASO Seen | AUC: 87.3 -> 86.7 | Hurt |
| use GEAL instead of LASO* | LASO Unseen · LASO Unseen | aIoU: 15.6 -> 16.7 | Helped |
| use GEAL instead of LASO under the listed point-cloud corruptions | PIAD-C Seen · PIAD-C Seen corruption types | aIoU | Helped |
| use only the 2D branch rather than the full GEAL model (ablation) | GEAL · PIAD Seen | aIoU: 22.5 -> 19.2 | Hurt |
| use only the 3D branch rather than the full GEAL model (ablation) | GEAL · PIAD Seen | aIoU: 22.5 -> 19.5 | Hurt |
| add the 2D-3D consistency alignment module (CAM) to the two branches (ablation) | GEAL · PIAD Seen | aIoU: 19.5 -> 22.0 | Helped |
| add GAFM (ablation) | GEAL · PIAD Seen | aIoU: 22.0 -> 22.5 | Helped |
| increase the number of rendered views from 6 to 12 without a view-dependent prompt (ablation) | GEAL · PIAD Seen | aIoU: 20.2 -> 21.4 | Helped |
| add a view-dependent prompt (ablation) | GEAL with r=112 and V=12 · PIAD Seen | aIoU: 21.4 -> 22.5 | Helped |
| increase the number of views from 12 to 14 (ablation) | GEAL with r=112 and a view-dependent prompt · PIAD Seen | aIoU: 22.5 -> 22.5 | No clear effect |
Limitations
- Gaussian covariance and opacity are manually set and fixed, not learned
- Unseen LASO split had to be reproduced from paper descriptions since the official split is unavailable
- PIAD lacks language annotations so questions were randomly assigned from LASO
Experimental setup
Datasets: LASO PIAD
Benchmarks: PIAD LASO PIAD-C LASO-C
Metrics: aIoU AUC SIM MAE
Baselines: LASO IAGNet MBDF PMF FRCNN ILN PFusion XMF ReferTrans ReLA 3D-SPS
Builds on: LASO IAGNet 3D Gaussian Splatting DINOv2
Closest papers by meaning
- SeqAfford: Sequential 3D Affordance Reasoning via Multimodal Large Language Model (CVPR 2025 · similarity 0.90)
- Learning 2D Invariant Affordance Knowledge for 3D Affordance Grounding (AAAI 2025 · similarity 0.90)
- 3D-AffordanceLLM: Harnessing Large Language Models for Open-Vocabulary Affordance Detection in 3D Worlds (ICLR 2025 · similarity 0.89)
- GREAT: Geometry-Intention Collaborative Inference for Open-Vocabulary 3D Object Affordance Grounding (CVPR 2025 · similarity 0.89)
- IAAO: Interactive Affordance Learning for Articulated Objects in 3D Environments (CVPR 2025 · similarity 0.88)
- AffordBot: 3D Fine-grained Embodied Reasoning via Multimodal Large Language Models (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.