Research map / 3D vision / 3D generation
OAKINK2: A Dataset of Bimanual Hands-Object Manipulation in Complex Task Completion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces OAKINK2, a multi-view bimanual hand-object manipulation dataset organized into affordance, primitive, and complex-task levels, and a Complex Task Completion framework using GPT-4 planning, oracle object retargeting, and a diffusion-based hand motion generator (MF-MDM).
Problem
Human-level understanding of bimanual object manipulation for long-horizon complex tasks lacks datasets that show both how affordances are fulfilled and how sub-actions depend on each other.
Approach
An object-centric three-level abstraction (affordance, primitive task, complex task with a primitive dependency graph) structures real mocap demonstrations, and a pipeline uses an LLM to decompose complex tasks into primitives and a task-aware diffusion model to generate hand motion per primitive.
Key results
- Multi-view POEM achieves 12.12 mm RR-MPJPE and 9.17 mm MPJPE in hand mesh reconstruction on OAKINK2 benchmark.
- MF-MDM generated hand motion reaches contact ratio 0.90, SIV 4.17 cm^3, and FID 1.369 on TaMF.
- Perceptual study gives generated hand motion a score of 4.66 ± 0.48 versus 3.64 ± 0.85 for ground truth.
Limitations
- Complex task completion evaluation of primitive planning is deferred to supplementary material
- Dataset is limited to four manipulation scenarios and 75 objects
Experimental setup
Datasets: OAKINK2
Benchmarks: OAKINK2
Metrics: MPJPE MPVPE PA-MPJPE AUC contact ratio solid intersection volume PSKL-J FID perceptual score
Baselines: METRO RLE KP-based Fit POEM
Builds on: OakInk MDM MoSH++ SMPL-X
Closest papers by meaning
- TACO: Benchmarking Generalizable Bimanual Tool-ACtion-Object Understanding (CVPR 2024 · similarity 0.89)
- BimArt: A Unified Approach for the Synthesis of 3D Bimanual Interaction with Articulated Objects (CVPR 2025 · similarity 0.89)
- GigaHands: A Massive Annotated Dataset of Bimanual Hand Activities (CVPR 2025 · similarity 0.88)
- InteractAnything: Zero-shot Human Object Interaction Synthesis via LLM Feedback and Object Affordance Parsing (CVPR 2025 · similarity 0.87)
- HOGSA: Bimanual Hand-Object Interaction Understanding with 3D Gaussian Splatting Based Data Augmentation (AAAI 2025 · similarity 0.87)
- ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learning (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.