Research map / Image generation / Autoregressive and latent generation

DexHandDiff: Interaction-aware Diffusion Planning for Adaptive Dexterous Manipulation

CVPR 2025 · Image generation › Autoregressive and latent generation · approach: Diffusion planning and policies · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DexHandDiff, a joint state-action classifier-guided diffusion planner with dynamics-based dual guidance and LLM-generated guidance functions to avoid ghost states and adapt dexterous manipulation to out-of-distribution goals.

Problem

Diffusion-based planners for dexterous, contact-rich manipulation produce unrealistic "ghost states" (objects moving without hand contact) and fail to adapt to goals outside the training distribution.

Approach

Jointly diffuse over concatenated state and action trajectories with a learned dynamics-consistency energy and a dual-phase (pre-contact alignment, post-contact goal-directed) guidance scheme, where guidance functions are generated by an LLM via text-to-reward-style prompting.

Key results

• Average success rate 59.2% on Door goal-adaptation tasks vs 29.5% for the best baseline (Diffuser guided sampling).
• Average 70.7% success across all goal-adaptive tasks in Table 3 vs 58.0% for Conditional DP and 34.0% for Diffuser (inpainting).
• Door Open 30° success 70.0% vs 16.7% for Diffusion Policy/Conditional DP.

Measured effects reported in the paper

Surprising findings

• Classifier-free Decision Diffuser and Diffusion Policy reach 100% on the training 90° door task but drop sharply on other angles (e.g., 0-30% at 30°).
• Learned dynamics often override guidance when the target angle is close to the training goal, causing failures (8 of 14 failures at 70° opened to 90°).

Limitations

• Learned dynamics can override guidance when goals are close to training data, causing failures.
• Evaluated only in simulation (Adroit and Shadow Hand); deployment with vision-estimated states is future work.
• LLM-generated guidance still requires about 5 rounds of human few-shot hint refinement; human-crafted guidance performs better.

Experimental setup

Datasets: D4RL Adroit demonstrations TQC+HER generated Shadow Hand block trajectories

Benchmarks: Adroit Door Adroit Pen Adroit Hammer Adroit Relocate Shadow Hand Block

Metrics: success rate L2 distance between predicted and simulated states control frequency

Baselines: Diffuser (goal inpainting) Diffuser (guided sampling) Decision Diffuser Diffusion Policy DexHandDiff-disc.

Builds on: Diffuser Decision Diffuser Diffusion Policy Eureka Text2Reward

Closest papers by meaning

• DexDeform: Dexterous Deformable Object Manipulation with Human Demonstrations and Differentiable Physics (ICLR 2023 · similarity 0.89)
• DexTrack: Towards Generalizable Neural Tracking Control for Dexterous Manipulation from Human References (ICLR 2025 · similarity 0.89)
• ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learning (CVPR 2025 · similarity 0.88)
• Scaffolding Dexterous Manipulation with Vision-Language Models (NeurIPS 2025 · similarity 0.88)
• Contact Map Transfer with Conditional Diffusion Model for Generalizable Dexterous Grasp Generation (NeurIPS 2025 · similarity 0.87)
• DexFlyWheel: A Scalable and Self-improving Data Generation Framework for Dexterous Manipulation (NeurIPS 2025 · similarity 0.87)