Research map / Vision-language and multimodal / Robot action and navigation
GarmentPile: Point-Level Visual Affordance Guided Retrieval and Adaptation for Cluttered Garments Manipulation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes point-level visual affordance learning with an affordance-guided pick-and-place adaptation module to retrieve garments from cluttered piles in simulation and the real world.
Problem
Retrieving garments from cluttered, entangled piles without contacting the floor or dragging other garments, where highly tangled piles may have no plausible manipulation point.
Approach
Dense per-point affordance (PointNet++-based) that encodes garment geometry, structure, and inter-garment relations for retrieval, plus a learned pick/place affordance pair that reorganizes entangled scenes until retrieval affordance is sufficiently high.
Key results
- Ours achieves 0.805 / 0.819 / 0.792 success rate in WashingMachine / Sofa / Basket simulation, versus best baseline Support-M 0.784 in Sofa and Where2Act 0.585 in WashingMachine.
- Removing adaptation drops success to 0.712 / 0.702 / 0.693 in the three scenes.
- Real-world success rates of 12/15, 13/15, 12/15 outperform Where2Act, Support-M and GPT-Fabric-M.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use point-level affordance guided retrieval and adaptation instead of Where2Act | cluttered garment manipulation · WashingMachine, Sofa, Basket | simulation success rate: 0.585 -> 0.805; 0.643 -> 0.819; 0.624 -> 0.792 | Helped |
| use point-level affordance guided retrieval and adaptation instead of Support-M | cluttered garment manipulation · WashingMachine, Sofa, Basket | simulation success rate: 0.562 -> 0.805; 0.784 -> 0.819; 0.684 -> 0.792 | Helped |
| use point-level affordance guided retrieval and adaptation instead of GPT-Fabric | cluttered garment manipulation · WashingMachine, Sofa, Basket | simulation success rate: 0.463 -> 0.805; 0.408 -> 0.819; 0.384 -> 0.792 | Helped |
| add the adaptation module (ablation) | cluttered garment manipulation · WashingMachine, Sofa, Basket | simulation success rate: 0.712 -> 0.805; 0.702 -> 0.819; 0.693 -> 0.792 | Helped |
| use learned pick affordance rather than random pick-point selection (ablation) | adaptation module · WashingMachine, Sofa, Basket | simulation success rate: 0.724 -> 0.805; 0.704 -> 0.819; 0.716 -> 0.792 | Helped |
| use learned place affordance rather than random place-point selection (ablation) | adaptation module · WashingMachine, Sofa, Basket | simulation success rate: 0.778 -> 0.805; 0.743 -> 0.819; 0.731 -> 0.792 | Helped |
Surprising findings
- GPT-Fabric-M, using GPT-4o for garment relation inference, performs worst (0.384-0.463 in simulation), attributed to GPT difficulty in cluttered relations.
Limitations
- Foundation segmentation models struggle to segment garments in occluded clutters, which hurts segmentation-based baselines.
- Real-world evaluation uses only 15 trials per scenario.
Experimental setup
Datasets: ClothesNet GarmentLab simulation scenes
Benchmarks: GarmentLab cluttered garment scenes (WashingMachine, Sofa, Basket)
Metrics: manipulation success rate
Baselines: Where2Act Support-M GPT-Fabric-M Ours w/o Adaptation
Builds on: PointNet++ Where2Act GPT-Fabric GarmentLab
Closest papers by meaning
- DexGarmentLab: Dexterous Garment Manipulation Environment with Generalizable Policy (NeurIPS 2025 · similarity 0.90)
- UniGarmentManip: A Unified Framework for Category-Level Garment Manipulation via Dense Visual Correspondence (CVPR 2024 · similarity 0.90)
- GarmentLab: A Unified Simulation and Benchmark for Garment Manipulation (NeurIPS 2024 · similarity 0.90)
- Learning Efficient Robotic Garment Manipulation with Standardization (ICML 2025 · similarity 0.88)
- Learning 3D Garment Animation from Trajectories of A Piece of Cloth (NeurIPS 2024 · similarity 0.87)
- DressWild: Feed-Forward Pose-Agnostic Garment Sewing Pattern Generation from In-the-Wild Images (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.