Research map / 3D vision / Human pose estimation
Forecasting of 3D Whole-body Human Poses with Grasping Objects
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes C3HOST, a graph-attention and MMD-aligned model that forecasts 55-joint whole-body (body plus both hands) 3D poses while grasping objects, evaluated on GRAB and BEHAVE.
Problem
Forecasting 3D human poses mainly covers major body joints and ignores fine-grained hand gestures and human-object interaction, which matter for HRI.
Approach
Treats body, left hand, and right hand as heterogeneous motion contexts, aligns them with maximum mean discrepancy, models inter-part dependencies with circular cross-attention, and fuses human and object features via a graph attention network with a gated human-object distance signal.
Key results
- On GRAB with objects, C3HOST reaches 106.8 mm average whole-body MPJPE at 1.0s, versus 119.7 mm for the best re-trained LTD baseline.
- On BEHAVE, C3HOST averages 69.7 mm versus 77.1 mm for LTD.
- Ablations show cross-context interaction lowers average error from 68.2 mm to 65.4 mm, and the gated sharing unit lowers it from 64.6 mm to 63.0 mm.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add cross-context interaction between body and hands (ablation) | C3HOST · GRAB, whole body | MPJPE (mm): 68.2 -> 65.4 | Helped |
| add cross-context alignment to cross-context interaction (ablation) | C3HOST · GRAB, whole body | MPJPE (mm): 65.4 -> 64.8 | Helped |
| add object shape features directly without aligning them to human features (ablation) | C3HOST · GRAB, whole body | MPJPE (mm): 64.8 -> 65.6 | Hurt |
| add cross-modal alignment of object and human features (ablation) | C3HOST with object features · GRAB, whole body | MPJPE (mm): 65.6 -> 64.6 | Helped |
| add the gated sharing unit for human-object distance (ablation) | C3HOST with cross-modal alignment · GRAB, whole body | MPJPE (mm): 64.6 -> 63.0 | Helped |
| use C3HOST instead of LTD | Whole-body forecasting · GRAB, whole body, 1.0s | MPJPE (mm): 119.7 -> 106.8 | Helped |
| use C3HOST instead of LTD | Whole-body forecasting · BEHAVE, whole body, 1.0s | MPJPE (mm): 131.7 -> 114.1 | Helped |
Surprising findings
- Directly concatenating object shape features into the model increased error (64.8 mm to 65.6 mm), attributed to feature distribution mismatch.
Limitations
- Baselines had to be re-trained because no whole-body forecasting methods exist, so comparisons depend on the authors' adaptation.
- BEHAVE has few long sequences (295 of 321 contain fewer than 60 frames), limiting evaluation.
- Only 1-second prediction horizons are evaluated.
Experimental setup
Datasets: GRAB BEHAVE
Benchmarks: GRAB BEHAVE
Metrics: MPJPE MPJPE-AW
Baselines: LTD DMGNN PGBIG SPGSN
Builds on: LTD PGBIG DMGNN SPGSN
Closest papers by meaning
- Expressive Forecasting of 3D Whole-Body Human Motions (AAAI 2024 · similarity 0.94)
- InterAct: Advancing Large-Scale Versatile 3D Human-Object Interaction Generation (CVPR 2025 · similarity 0.88)
- DiffGrasp: Whole-Body Grasping Synthesis Guided by Object Motion Using a Diffusion Model (AAAI 2025 · similarity 0.87)
- Multimodal Sense-Informed Forecasting of 3D Human Motions (CVPR 2024 · similarity 0.87)
- CoDA: Coordinated Diffusion Noise Optimization for Whole-Body Manipulation of Articulated Objects (NeurIPS 2025 · similarity 0.87)
- MotionMap: Representing Multimodality in Human Pose Forecasting (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.