Research map / Image generation / Autoregressive and latent generation
AffordDP: Generalizable Diffusion Policy with Transferable Affordance
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AffordDP, a diffusion policy conditioned on transferable 3D affordances (contact point plus post-contact trajectory) transferred to unseen objects via foundation-model correspondences and ICP, with adaptive affordance-guided diffusion sampling.
Problem
Diffusion-based visuomotor policies generalize poorly beyond the training category and similar appearances, failing on unseen object instances and categories in manipulation.
Approach
Represent affordances as 3D contact points and post-contact trajectories, transfer them from in-domain objects to unseen targets using SD-DINOv2 semantic correspondences and ICP-based 6D transforms, condition the diffusion policy on them, and steer sampling with an end-effector-to-contact-point distance guidance term.
Key results
- In simulation unified training, AffordDP reaches 73.3% success on unseen-category PullDrawer vs 3.3% for DP and 3.3% for DP3.
- In real-world experiments, AffordDP achieves 40.0% on unseen-category PullDrawer, 50.0% on unseen-category OpenDoor, and 50.0% on unseen-category Pick&Place, where DP fails (0%) across all tasks.
- Ablation shows contact-point-only reaches 17.8% on unseen categories, adding trajectory 22.2%, and adding affordance guidance 26.7%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use transferable affordance conditioning and affordance-guided sampling instead of DP3 | Diffusion Policy on PullDrawer · Simulation, unseen category | success rate: 3.3% -> 73.3% | Helped |
| Use transferable affordance conditioning and affordance-guided sampling instead of DP3 | Diffusion Policy on OpenDoor · Simulation, unseen category | success rate: 5.6% -> 26.7% | Helped |
| Use transferable affordance conditioning and affordance-guided sampling instead of DP3 | Diffusion Policy on PullDrawer · Real world, unseen instance | success rate: 0.0% -> 45.0% | Helped |
| Use transferable affordance conditioning and affordance-guided sampling instead of DP3 | Diffusion Policy on OpenDoor · Real world, unseen category | success rate: 30.0% -> 50.0% | Helped |
| Use transferable affordance conditioning and affordance-guided sampling instead of DP3 | Diffusion Policy on Pick & Place · Real world, unseen instance | success rate: 30.0% -> 65.0% | Helped |
| Add the dynamic affordance trajectory to contact-point-only conditioning (ablation) | AffordDP · Simulation, unseen instance | success rate: 42.5% -> 50.0% | Helped |
| Add the dynamic affordance trajectory to contact-point-only conditioning (ablation) | AffordDP · Simulation, seen instance | success rate: 72.2% -> 71.6% | Hurt |
| Add adaptive affordance guidance to contact-point-and-trajectory conditioning (ablation) | AffordDP · Simulation, unseen category | success rate: 22.2% -> 26.7% | Helped |
| Add adaptive affordance guidance to contact-point-and-trajectory conditioning (ablation) | AffordDP · Simulation, seen instance | success rate: 71.6% -> 74.4% | Helped |
Surprising findings
- DP fails in all real-world tasks (0% success), consistent with needing over 200 demonstrations.
- Simulation object-specific results show baselines comparable to AffordDP under low variance, with the gap appearing mainly at higher variance.
Limitations
- Foundation models have limited spatial understanding, restricting part-specific affordance transfer.
- Effectiveness is limited where modeling or extracting affordances is challenging.
- Evaluated on only two simulated tasks and three real-world tasks with few demonstrations.
Experimental setup
Datasets: GAPartNet
Benchmarks: IsaacGym PullDrawer IsaacGym OpenDoor Real-world PullDrawer Real-world OpenDoor Real-world Pick&Place
Metrics: success rate
Baselines: Diffusion Policy DP3
Builds on: Diffusion Policy 3D Diffusion Policy DDIM Point-SAM Grounded-SAM
Closest papers by meaning
- Zero-Shot Robotic Manipulation with Pre-Trained Image-Editing Diffusion Models (ICLR 2024 · similarity 0.90)
- Generate Subgoal Images before Act: Unlocking the Chain-of-Thought Reasoning in Diffusion Model for Robot Manipulation with Multimodal Prompts (CVPR 2024 · similarity 0.89)
- GravMAD: Grounded Spatial Value Maps Guided Action Diffusion for Generalized 3D Manipulation (ICLR 2025 · similarity 0.89)
- Affordance-R1: Reinforcement Learning for Generalizable Affordance Reasoning in Multimodal Large Language Models (AAAI 2026 · similarity 0.89)
- Exploring the Limits of Vision-Language-Action Manipulation in Cross-task Generalization (NeurIPS 2025 · similarity 0.89)
- VidBot: Learning Generalizable 3D Actions from In-the-Wild 2D Human Videos for Zero-Shot Robotic Manipulation (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.