Research map / 3D vision / 3D generation
BimArt: A Unified Approach for the Synthesis of 3D Bimanual Interaction with Articulated Objects
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Generates 3D bimanual hand motions for articulated objects from object trajectories by first predicting distance-based contact maps with an articulation-aware part-based BPS representation, then using them to guide a diffusion motion model followed by MANO optimization.
Problem
Synthesizing diverse and physically plausible bimanual hand-object interactions with articulated objects given only object trajectories (global motion and articulation), without reference grasps or coarse hand trajectories.
Approach
A normalized part-based basis point set (BPS) object representation in a canonical frame, plus a two-stage diffusion pipeline where a contact-map diffusion model provides an intermediate contact prior that guides a hand-motion diffusion model via contact-map discrepancy guidance, followed by optimization-based MANO fitting.
Key results
- On ARCTIC, BimArt reduces penetration to 2.03% vs 42.52% for CAMS-B and 30.44% for OMOMO-B, with contact 99.63% and articulation 85.57%.
- On HOI4D under the unified setting, BimArt achieves an articulation score of 0.887 on scissors with optimization and outperforms CAMS-X in contact and articulation consistency by a large margin.
- In a user study with 55 respondents, participants preferred BimArt motions over MDM-B and OMOMO-B for all objects (p < 10^-3).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use BimArt rather than MDM-B for bimanual motion synthesis | ARCTIC · ARCTIC test | Pen 1cm (%): 66.71 -> 2.0346 | Helped |
| use BimArt rather than OMOMO-B for bimanual motion synthesis | ARCTIC · ARCTIC test | Pen 1cm (%): 30.435 -> 2.0346 | Helped |
| use normalized part-based BPS instead of unnormalized BPS (ablation) | BimArt · ARCTIC test | Mul (cm): 6.2551 -> 6.97928 | Helped |
| use normalized part-based BPS instead of normalized part-agnostic BPS (ablation) | BimArt · ARCTIC test | Mul (cm): 6.317 -> 6.97928 | Helped |
| use the proposed surface-keypoint and direction-vector hand representation instead of MANO-Rep (ablation) | BimArt · ARCTIC test | Pen 1cm (%): 22.201 -> 20.273 | Helped |
| remove global states G (ablation) | BimArt · ARCTIC test | Pen 1cm (%): 20.273 -> 20.414 | Hurt |
| add contact conditioning (ablation) | BimArt · ARCTIC test | Art (%): 79.591 -> 83.089 | Helped |
| add contact-map guidance (ablation) | BimArt · ARCTIC test | CM (cm): 1.1505 -> 1.1284 | Helped |
| add optimization-based refinement (ablation) | BimArt · ARCTIC test | Pen 1cm (%): 16.496 -> 2.0346 | Helped |
Surprising findings
- CAMS-B achieves higher multi-modality (8.56 cm) and lower acceleration than BimArt yet has far higher penetration (42.5%), showing diversity metrics alone are misleading.
Limitations
- Restricted to object categories present in ARCTIC and HOI4D; no zero-shot generalization to open-vocabulary objects.
- Slow diffusion sampling; suggested faster samplers like DDIM or latent diffusion as future improvements.
- Only two-part articulated objects with one revolute joint (7 DoF) are handled.
Experimental setup
Datasets: ARCTIC HOI4D
Benchmarks: ARCTIC HOI4D
Metrics: multi-modality (Mul) acceleration penetration 1cm contact percentage articulation percentage contact map discrepancy user preference rate
Baselines: CAMS-B MDM-B OMOMO-B CAMS-X GraspTTA ManipNet MDM-U OMOMO-U
Builds on: DDPM MDM CAMS OMOMO Basis Point Sets
Closest papers by meaning
- OAKINK2: A Dataset of Bimanual Hands-Object Manipulation in Complex Task Completion (CVPR 2024 · similarity 0.89)
- HOGSA: Bimanual Hand-Object Interaction Understanding with 3D Gaussian Splatting Based Data Augmentation (AAAI 2025 · similarity 0.88)
- GigaHands: A Massive Annotated Dataset of Bimanual Hand Activities (CVPR 2025 · similarity 0.87)
- How Do I Do That? Synthesizing 3D Hand Motion and Contacts for Everyday Interactions (CVPR 2025 · similarity 0.87)
- TACO: Benchmarking Generalizable Bimanual Tool-ACtion-Object Understanding (CVPR 2024 · similarity 0.87)
- ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learning (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.