Research map / Image generation / Autoregressive and latent generation
GeneOH Diffusion: Towards Generalizable Hand-Object Interaction Denoising via Denoising Diffusion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GeneOH Diffusion, a contact-centric hand-object interaction representation plus a canonical diffusion denoiser applied via "denoising via diffusion" in three progressive stages, to clean noisy hand trajectories with strong cross-domain generalization.
Problem
Denoising hand-object interaction sequences, i.e., refining erroneous hand trajectories (penetrations, unnatural poses, inconsistent contacts) so they are realistic, while generalizing to unseen objects, interactions and noise patterns.
Approach
A contact-centric HOI representation (GeneOH) canonicalized by generalized contact points, plus a canonical denoiser trained on a whitened noise space; at test time the noisy input is diffused to that space by a chosen timestep and then cleaned, applied progressively over hand keypoints, spatial relations, and temporal relations.
Key results
- On GRAB, GeneOH Diffusion reaches MPJPE 9.28 mm and IV 1.23 cm^3 versus 12.23 mm and 1.94 cm^3 for the best TOCH variant.
- When trained only on GRAB, it reduces HOI4D intersection volume to 1.99 cm^3 versus 4.09 for TOCH, and HO motion consistency to 9.81 versus 25.67 for TOCH (w/ MixStyle).
- On ARCTIC, MPJPE is 11.57 mm versus 13.82 mm for the best baseline.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use GeneOH Diffusion instead of the strongest listed baseline | HOI denoising · GRAB | IV (cm³): 1.94 -> 1.23 | Helped |
| use GeneOH Diffusion instead of the strongest listed baseline | HOI denoising · GRAB (Beta) | IV (cm³): 1.52 -> 1.19 | Helped |
| use GeneOH Diffusion instead of the strongest listed baseline | HOI denoising · HOI4D | HO Motion Consistency (mm²): 25.67 -> 9.81 | Helped |
| use GeneOH Diffusion instead of the strongest listed baseline | HOI denoising · ARCTIC | HO Motion Consistency (mm²): 18.02 -> 0.92 | Helped |
| add SpatialDiff and TemporalDiff (ablation) | GeneOH Diffusion · HOI4D | IV (cm³): 2.94 -> 1.99 | Helped |
| add SpatialDiff and TemporalDiff (ablation) | GeneOH Diffusion · HOI4D | Penetration Depth (mm): 3.45 -> 2.15 | Helped |
| add SpatialDiff and TemporalDiff (ablation) | GeneOH Diffusion · HOI4D | HO Motion Consistency (mm²): 31.67 -> 9.81 | Helped |
| add TemporalDiff (ablation) | GeneOH Diffusion · HOI4D | IV (cm³): 1.72 -> 1.99 | Hurt |
| add TemporalDiff (ablation) | GeneOH Diffusion · HOI4D | Penetration Depth (mm): 1.90 -> 2.15 | Hurt |
| add TemporalDiff (ablation) | GeneOH Diffusion · HOI4D | HO Motion Consistency (mm²): 34.25 -> 9.81 | Helped |
| use denoising via diffusion rather than an autoencoder (ablation) | GeneOH Diffusion · HOI4D | IV (cm³): 3.16 -> 1.99 | Helped |
| use denoising via diffusion rather than an autoencoder (ablation) | GeneOH Diffusion · HOI4D | Penetration Depth (mm): 3.83 -> 2.15 | Helped |
Surprising findings
- Ablation without the canonicalization (w/o Canon.) gives a better HO motion consistency (13.26) than the full model on HOI4D, though the full model is best on the other key metrics.
Limitations
- Assumes accurate object pose trajectories, which may not hold for in-the-wild video estimates.
- Stochastic denoiser yields multiple solutions, so results depend on selecting the closest sample out of 100 seeds.
Experimental setup
Datasets: GRAB ARCTIC HOI4D
Benchmarks: GRAB GRAB (Beta) HOI4D ARCTIC
Metrics: MPJPE MPVPE C-IoU solid intersection volume penetration depth proximity error HO motion consistency
Baselines: TOCH TOCH (w/ MixStyle) TOCH (w/ Aug.)
Builds on: TOCH denoising diffusion probabilistic models MixStyle
Closest papers by meaning
- G-HOP: Generative Hand-Object Prior for Interaction Reconstruction and Grasp Synthesis (CVPR 2024 · similarity 0.86)
- HOIDiffusion: Generating Realistic 3D Hand-Object Interaction Data (CVPR 2024 · similarity 0.86)
- HOI-Swap: Swapping Objects in Videos with Hand-Object Interaction Awareness (NeurIPS 2024 · similarity 0.86)
- InterHandGen: Two-Hand Interaction Generation via Cascaded Reverse Diffusion (CVPR 2024 · similarity 0.86)
- CoDA: Coordinated Diffusion Noise Optimization for Whole-Body Manipulation of Articulated Objects (NeurIPS 2025 · similarity 0.86)
- LatentHOI: On the Generalizable Hand Object Motion Generation with Latent Hand Diffusion. (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.