Research map / Image generation / Fast diffusion sampling
InterHandGen: Two-Hand Interaction Generation via Cascaded Reverse Diffusion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes InterHandGen, a diffusion-based generative prior for two-hand interaction that factorizes the joint two-hand distribution into unconditional and conditional single-hand diffusion models, trained with one shared network via conditioning dropout.
Problem
Generating plausible and diverse interacting two-hand shapes (with or without an object) is under-explored, and directly modeling the joint distribution of two hands is combinatorially complex, leading to suboptimal generation.
Approach
Decompose the joint two-hand distribution as p(xl)p(xr|xl), train a single transformer diffusion model with conditioning dropout and left-right mirroring to learn both factors, sample cascaded (anchor hand then interacting hand) with classifier-free guidance and an anti-penetration guidance, and use the model as a frozen SDS-style regularizer for monocular two-hand reconstruction.
Key results
- On two-hand generation (InterHand2.6M), Ours reaches FHID 1.00 and recall 0.85 versus 3.48 FHID and 0.47 recall for BUDDI*.
- On object-conditioned generation (ARCTIC), Ours reaches FHID 12.91 versus 21.75 for the VAE baseline and 22.56 for ContactGen*.
- As a reconstruction prior on InterWild, MRRPE improves from 29.29 to 26.56 on InterHand2.6M and from 31.35 to 26.63 on HIC.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add decomposed unconditional and conditional single-hand modeling (ablation) | InterHandGen · InterHand2.6M test, two-hand synthesis | FHID: 2.09 -> 1.00 | Helped |
| Use a shared network for conditional and unconditional distributions rather than separate networks (ablation) | InterHandGen · InterHand2.6M test, two-hand synthesis | FHID: 1.32 -> 1.00 | Helped |
| Add self-attention (ablation) | InterHandGen · InterHand2.6M test, two-hand synthesis | FHID: 2.87 -> 1.00 | Helped |
| Add self-attention (ablation) | InterHandGen · InterHand2.6M test, two-hand synthesis | Diversity: 3.16 -> 3.59 | Helped |
| Add classifier-free guidance (ablation) | InterHandGen · InterHand2.6M test, two-hand synthesis | FHID: 1.12 -> 1.00 | Helped |
| Add classifier-free guidance (ablation) | InterHandGen · InterHand2.6M test, two-hand synthesis | Diversity: 3.61 -> 3.59 | No clear effect |
| Add anti-penetration guidance (ablation) | InterHandGen · InterHand2.6M test, two-hand synthesis | PenVol: 6.58 -> 0.76 | Helped |
| Add anti-penetration guidance (ablation) | InterHandGen · InterHand2.6M test, two-hand synthesis | ProxRatio: 0.97 -> 0.97 | No clear effect |
| Add InterHandGen prior to InterWild | InterWild · InterHand2.6M test, monocular reconstruction | MPVPE: 13.01 -> 12.10 | Helped |
| Add InterHandGen prior to InterWild | InterWild · InterHand2.6M test, monocular reconstruction | MPRPE: 29.29 -> 26.56 | Helped |
| Add InterHandGen prior to InterWild | InterWild · HIC, monocular reconstruction | MPRPE: 31.35 -> 26.63 | Helped |
Surprising findings
- Directly modeling the joint two-hand distribution yields suboptimal generation compared with the factorized formulation.
- ContactGen adapted to two hands biases towards heavy-contact cases and performs poorly on general two-hand-object interaction.
Limitations
- Temporal dimension is not modeled; only static two-hand shapes are generated.
- Evaluation metrics rely on a newly trained PointNet++ feature backbone rather than an established benchmark.
Experimental setup
Datasets: InterHand2.6M ARCTIC MSCOCO HIC
Benchmarks: InterHand2.6M ARCTIC HIC
Metrics: FHID KHID diversity precision recall penetration volume MPVPE MPJPE MRRPE
Baselines: VAE (Zuo et al.) BUDDI* ContactGen* InterWild Ours w/o Decomposition Ours w/o Shared Network
Builds on: DDIM classifier-free guidance Score Distillation Sampling BUDDI MANO
Closest papers by meaning
- HandDiffuse: Generative Controllers for Two-Hand Interactions via Diffusion Models (AAAI 2025 · similarity 0.90)
- G-HOP: Generative Hand-Object Prior for Interaction Reconstruction and Grasp Synthesis (CVPR 2024 · similarity 0.89)
- FoundHand: Large-Scale Domain-Specific Learning for Controllable Hand Image Generation (CVPR 2025 · similarity 0.89)
- Single-view Image to Novel-view Generation for Hand-Object Interactions (AAAI 2025 · similarity 0.88)
- The Surprising Effectiveness of Video Diffusion Models for Hand Motion Reconstruction (NeurIPS 2026 · similarity 0.88)
- HandBooster: Boosting 3D Hand-Mesh Reconstruction by Conditional Synthesis and Sampling of Hand-Object Interactions (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.