Research map / 3D vision / 3D generation
CoDA: Coordinated Diffusion Noise Optimization for Whole-Body Manipulation of Articulated Objects
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Synthesizes whole-body manipulation of articulated objects by optimizing diffusion noise over three separately trained diffusion models (body, left hand, right hand), guided by an end-effector basis-point-set representation shared with object geometry.
Problem
Generating coordinated body, hand, and articulated object motion for whole-body manipulation, requiring tight body-hand coordination and precise finger placement on object parts.
Approach
Perform noise-space optimization over three decoupled diffusion models (body, left hand, right hand) so gradients flow through the kinematic chain to coordinate posture with hand targets, and represent end-effectors as distances to the same BPS used for object geometry.
Key results
- On ARCTIC, achieves FID 2.283 and R-Precision 0.477 versus best baseline OMOMO (3.710, 0.406), with IV 5.25 vs 12.72 for the second-best.
- On GRAB, achieves FID 21.544 vs 25.017 for OMOMO, and interpenetration volume 4.93 vs 9.31 for CHOIS.
- User study with 16 participants: 88.7% best motion realism rate and 87.3% best physical plausibility rate versus baselines.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use separate models for object motion and end-effector trajectories rather than one joint model (ablation) | CoDA · ARCTIC | FID: 3.790 -> 2.283 | Helped |
| Represent end-effector trajectories with end-effector BPS instead of object-relative coordinates (ablation) | CoDA · ARCTIC | FID: 4.069 -> 2.283 | Helped |
| Use RoPE-based object-motion representation instead of object velocity and rotational velocity inputs (ablation) | CoDA · ARCTIC | FID: 2.714 -> 2.283 | Helped |
| Optimize diffusion noise instead of using conditional diffusion with fingertip trajectories as input (ablation) | CoDA · ARCTIC | FID: 4.883 -> 2.283 | Helped |
| Use decoupled body-hand diffusion models instead of one model for the entire body (ablation) | CoDA · ARCTIC | FID: 2.699 -> 2.283 | Helped |
| Include AMASS in body-motion-model training (ablation) | CoDA · ARCTIC | FID: 3.305 -> 2.283 | Helped |
| Use CoDA rather than OMOMO | whole-body manipulation generation · ARCTIC | FID: 3.710 -> 2.283 | Helped |
| Use CoDA rather than OMOMO | whole-body object interaction generation · GRAB | FID: 25.017 -> 21.544 | Helped |
Surprising findings
- Removing the optimization step (variant d) raised foot skating to 0.030 and interpenetration volume to 16.39, showing optimization is key to physical plausibility.
Limitations
- Optimization is slow (about 17 minutes per sequence), limiting real-time use.
- Generalization to novel object categories is limited by small object diversity in ARCTIC and GRAB.
- Handles only single-object rotational articulation; no obstacle avoidance or multi-object scenes.
Experimental setup
Datasets: ARCTIC GRAB AMASS
Benchmarks: ARCTIC GRAB
Metrics: FID R-Precision Diversity Foot skating Interpenetration volume Interpenetration depth Contact ratio user study
Baselines: IMoS MDM Text2HOI OMOMO CHOIS
Builds on: DNO DOODL MDM CaPE CHOIS
Closest papers by meaning
- DiffGrasp: Whole-Body Grasping Synthesis Guided by Object Motion Using a Diffusion Model (AAAI 2025 · similarity 0.91)
- Optimizing Diffusion Noise Can Serve As Universal Motion Priors (CVPR 2024 · similarity 0.88)
- LatentHOI: On the Generalizable Hand Object Motion Generation with Latent Hand Diffusion. (CVPR 2025 · similarity 0.87)
- InterMimic: Towards Universal Whole-Body Control for Physics-Based Human-Object Interactions (CVPR 2025 · similarity 0.87)
- G-HOP: Generative Hand-Object Prior for Interaction Reconstruction and Grasp Synthesis (CVPR 2024 · similarity 0.87)
- Forecasting of 3D Whole-body Human Poses with Grasping Objects (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.