Research map / Detection and segmentation / Open-set and few-shot detection
HOI-M^3: Capture Multiple Humans and Objects Interaction within Contextual Environment
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces HOI-M3, a large multi-human multi-object motion capture dataset (IMU-equipped objects, 42 cameras), and two baselines for monocular multi-HOI capture and diffusion-based multi-HOI generation.
Problem
Modeling human-object interactions involving multiple humans and multiple objects in contextual environments is held back by a lack of 3D motion capture data.
Approach
A dense hybrid capture setup (42 synchronized cameras plus IMUs embedded in pre-scanned objects) with an inertial-aided joint optimization yields accurate 3D tracking of multiple humans and objects; the paper pairs it with a one-stage monocular multi-HOI capture network and a conditional diffusion generator over multi-person, multi-object parameters.
Key results
- Monocular multi-HOI capture reaches 68.5 PCKrel and 235.0 object Chamfer distance, versus 43.9 PCKrel for PHOSA and 10.4 for CHORE on HOI-M3.
- Multi-HOI generation achieves FID 36.906 for joint evaluation and Pene 9.265% on HOI-M3 (separated evaluation FID 16.502 for people and 10.609 for objects).
- HOI-M3 contains 199 sequences, 181M frames, 42 cameras, 90 objects, and 31 to 46 subjects (paper gives both counts) over about 20 hours.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the monocular multiple-HOI capture method instead of PHOSA | HOI-M3 capture · HOI-M3, all predictions | PCKrel: 43.9 -> 68.5 | Helped |
| Use the monocular multiple-HOI capture method instead of CHORE | HOI-M3 capture · HOI-M3, all predictions | PCKrel: 10.4 -> 68.5 | Helped |
| Use the monocular multiple-HOI capture method instead of PHOSA | HOI-M3 capture · HOI-M3, all predictions | Chamfero: 1454.3 -> 235.0 | Helped |
| Use the monocular multiple-HOI capture method instead of CHORE | HOI-M3 capture · HOI-M3, all predictions | Chamfero: 465.8 -> 235.0 | Helped |
| Use the monocular multiple-HOI capture method instead of PHOSA | HOI-M3 capture · HOI-M3, all predictions | V2V: 691.4 -> 297.8 | Helped |
| Use the monocular multiple-HOI capture method instead of CHORE | HOI-M3 capture · HOI-M3, all predictions | V2V: 340.2 -> 297.8 | Helped |
Surprising findings
- Current single-HOI capture methods perform poorly when applied to multi-person multi-object scenes, with CHORE reaching only 10.4 PCKrel.
Limitations
- Capture is limited to indoor settings with fixed illumination and few background variations.
- Only 5 common scenes are covered.
- Generated and captured multi-HOI motions remain far from vivid, indicating the tasks are still challenging.
Experimental setup
Datasets: HOI-M3 BEHAVE InterCap
Benchmarks: HOI-M3
Metrics: PCK relative PCK absolute (3DPCKabs) Chamfer distance v2v FID Pene
Baselines: PHOSA CHORE
Builds on: SMPL PHOSA CHORE Ego-Ego DDPM
Closest papers by meaning
- InterAct: Advancing Large-Scale Versatile 3D Human-Object Interaction Generation (CVPR 2025 · similarity 0.92)
- Learning to Generate Human-Human-Object Interactions from Textual Descriptions (NeurIPS 2025 · similarity 0.90)
- Learning Human-Object Interaction as Groups (NeurIPS 2025 · similarity 0.90)
- Reconstructing In-the-Wild Open-Vocabulary Human-Object Interactions (CVPR 2025 · similarity 0.90)
- HORP: Human-Object Relation Priors Guided HOI Detection (CVPR 2025 · similarity 0.89)
- Real2Sim in HOI: Toward Physically Plausible HOI Reconstruction from Monocular Videos (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.