Research map / 3D vision / Human pose estimation
I'M HOI: Inertia-aware Monocular Capture of 3D Human-Object Interactions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Captures 3D human and object motion from a monocular RGB video plus one object-mounted IMU, via end-to-end holistic tracking followed by a category-aware motion diffusion refinement, and contributes the IMHD2 dataset.
Problem
Capturing 3D human-object interactions from a minimal hybrid setting (monocular RGB plus object-mounted IMU) is hard because RGB alone suffers depth ambiguity and occlusion, and prior methods are slow and human-only IMU fusion ignores objects.
Approach
Treats the object as an extra body joint for end-to-end holistic tracking, fuses IMU signals via mesh-aligned feedback loops, and refines results with a category-specific conditional diffusion model using an over-parameterized human-hand-object representation with IMU-based regularization.
Key results
- On IMHD2 (fast motions), I'm-HOI reaches 6.50 cm smpl and 6.93 cm object per-frame Chamfer distance, versus 19.96/23.28 for VisTracker.
- On BEHAVE per-frame, I'm-HOI gets 5.66/8.92 cm versus VisTracker's 6.76/10.32 cm; generalization on HODome per-frame is 8.19/9.05 cm vs VisTracker's 11.87/19.86 cm.
- Runtime is about 0.5 s per frame, versus about 20 s per frame for VisTracker and about 2 minutes per frame for PHOSA.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use I’m-HOI instead of VisTracker | Human-object tracking · IMHD2 (fast) | CD (per-frame), smpl: 19.96 -> 6.50 | Helped |
| Use I’m-HOI instead of VisTracker | Human-object tracking · IMHD2 (fast) | CD (per-frame), object: 23.28 -> 6.93 | Helped |
| Use I’m-HOI instead of VisTracker | Human-object tracking · BEHAVE (slow to medium) | CD (per-frame), smpl: 5.25 -> 5.26 | No clear effect |
| Use I’m-HOI instead of VisTracker | Human-object tracking · InterCap | CD (10s), object: 11.38 -> 7.14 | Helped |
| Use I’m-HOI instead of VisTracker | Human-object tracking · HODome | CD (per-frame), object: 19.86 -> 9.05 | Helped |
| Use I’m-HOI instead of VisTracker | Human-object tracking · CHAIRS | CD (per-frame), smpl: 17.42 -> 9.55 | Helped |
| Add mesh alignment feedback (maf.) (ablation) | I’m-HOI · IMHD2 | CD (per-frame), object: 15.12 -> 9.73 | Helped |
| Add optimization module (optim.) (ablation) | I’m-HOI with mesh alignment feedback · IMHD2 | CD (10s), object: 19.87 -> 10.75 | Helped |
| Add diffusion filter (filter.) (ablation) | I’m-HOI with mesh alignment feedback and optimization · IMHD2 | CD (per-frame), smpl: 7.16 -> 6.50 | Helped |
| Add diffusion filter (filter.) (ablation) | I’m-HOI with mesh alignment feedback and optimization · IMHD2 | CD (10s), object: 10.75 -> 8.53 | Helped |
| Add IMU modality (ablation) | I’m-HOI · IMHD2 | CD (per-frame), object: 14.49 -> 6.93 | Helped |
| Add IMU modality (ablation) | I’m-HOI · IMHD2 | CD (per-frame), smpl: 10.58 -> 6.50 | Helped |
Surprising findings
- Naively adding the IMU optimization term to baselines (PHOSA, CHORE, VisTracker) gives only modest gains, not matching the proposed network design.
Limitations
- Relies on pre-scanned object templates and manual sensor-to-template coordinate alignment.
- Restricted to rigid object tracking; articulated or deformable objects are not handled.
Experimental setup
Datasets: BEHAVE InterCap IMHD2 HODome CHAIRS
Benchmarks: IMHD2 BEHAVE InterCap HODome CHAIRS
Metrics: per-frame Chamfer distance (cm) sliding-window (10s) Chamfer distance (cm)
Baselines: PHOSA CHORE VisTracker
Builds on: SMPL CHORE VisTracker PHOSA MotionDiffuse
Closest papers by meaning
- InterAct: Advancing Large-Scale Versatile 3D Human-Object Interaction Generation (CVPR 2025 · similarity 0.89)
- HOI-M^3: Capture Multiple Humans and Objects Interaction within Contextual Environment (CVPR 2024 · similarity 0.88)
- Real2Sim in HOI: Toward Physically Plausible HOI Reconstruction from Monocular Videos (NeurIPS 2026 · similarity 0.88)
- Motions as Queries: One-Stage Multi-Person Holistic Human Motion Capture (CVPR 2025 · similarity 0.87)
- SViMo: Synchronized Diffusion for Video and Motion Generation in Hand-object Interaction Scenarios (NeurIPS 2025 · similarity 0.87)
- InterMimic: Towards Universal Whole-Body Control for Physics-Based Human-Object Interactions (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.