Research map / Detection and segmentation / 3D detection and driving
LASO: Language-guided Affordance Segmentation on 3D Object
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces LASO, a task of segmenting 3D object parts from natural-language affordance questions, with a 19,751-pair dataset and a baseline model PointRefer (adaptive fusion + question-conditioned dynamic kernels).
Problem
Existing 3D affordance segmentation is visual-only with predefined affordance types, which limits generalization to unseen objects and integration with LLM task planners that issue language-based affordance instructions.
Approach
Segment the affordance part of a 3D point cloud directly from a free-form natural-language question, without auxiliary affordance-label prediction, using an adaptive multi-scale text-point fusion module and question-conditioned affordance queries that generate dynamic convolution kernels for mask prediction.
Key results
- PointRefer achieves 20.8 mIoU, 87.3 AUC, 0.629 SIM and 0.093 MAE on the seen split, beating the best baseline IAGNet (17.8 mIoU).
- On the unseen split PointRefer reaches 14.6 mIoU and 80.2 AUC, versus 12.9 mIoU for IAGNet.
- Blind test without question input drops mIoU to 10.8, showing the model relies on the question.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use PointRefer instead of IAGNet | LASO · LASO Seen | mIoU: 17.8 -> 20.8 | Helped |
| use PointRefer instead of IAGNet | LASO · LASO Unseen | mIoU: 12.9 -> 14.6 | Helped |
| add question information (ablation) | PointRefer · LASO Seen | mIoU: 10.8 -> 20.8 | Helped |
| add AFM and PRD (ablation) | PointRefer · LASO Seen | mIoU: 17.7 -> 20.8 | Helped |
| add AFM (ablation) | PointRefer · LASO Seen | mIoU: 19.8 -> 20.8 | Helped |
| use multi-scale rather than single-scale AFM (ablation) | PointRefer · LASO Seen | mIoU: 20.4 -> 20.8 | Helped |
| add PRD (ablation) | PointRefer · LASO Seen | mIoU: 18.7 -> 20.8 | Helped |
| condition affordance queries on the question (ablation) | PointRefer · LASO Seen | mIoU: 19.2 -> 20.8 | Helped |
| use dynamic filtering rather than IAGNet mask prediction (ablation) | PointRefer · LASO Seen | mIoU: 19.5 -> 20.8 | Helped |
| replace PointNet++ with PointMLP (ablation) | PointRefer · LASO Seen | mIoU: 18.2 -> 20.8 | Hurt |
| replace RoBERTa with DeBERTa (ablation) | PointRefer · LASO Unseen | mIoU: 14.5 -> 14.6 | No clear effect |
Surprising findings
- Image-based referred segmentation methods (ReferTrans, ReLA) underperform the 3D-specific IAGNet, which the authors attribute to the modality gap and context-rich questions versus short labels in image-text pretraining.
Limitations
- Performance drops substantially on unseen object-affordance pairings (14.6 vs 20.8 mIoU).
- Failure cases include overly brief questions and over-segmentation when multiple disjoint affordance parts are targeted.
- Dataset is built from 58 object-affordance combinations with 870 questions, limiting diversity.
Experimental setup
Datasets: 3D-AffordanceNet LASO dataset (question-point pairs)
Benchmarks: LASO
Metrics: mIoU AUC SIM MAE
Baselines: ReferTrans ReLA 3D-SPS IAGNet
Builds on: 3D AffordanceNet IAGNet PointNet++ Mask2Former-style learnable queries
Closest papers by meaning
- 3D-AffordanceLLM: Harnessing Large Language Models for Open-Vocabulary Affordance Detection in 3D Worlds (ICLR 2025 · similarity 0.91)
- SeqAfford: Sequential 3D Affordance Reasoning via Multimodal Large Language Model (CVPR 2025 · similarity 0.90)
- GREAT: Geometry-Intention Collaborative Inference for Open-Vocabulary 3D Object Affordance Grounding (CVPR 2025 · similarity 0.89)
- Grounding 3D Object Affordance with Language Instructions, Visual Observations and Interactions (CVPR 2025 · similarity 0.89)
- UniFunc3D: Unified Active Spatial-Temporal Grounding for 3D Affordance Segmentation (NeurIPS 2026 · similarity 0.88)
- TB-HSU: Hierarchical 3D Scene Understanding with Contextual Affordances (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.