Research map / 3D vision / 3D generation
ParaHome: Parameterizing Everyday Home Activities Towards 3D Generative Modeling of Human-Object Interactions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces the ParaHome multi-view + IMU/glove capture system and a home-activity human-object interaction dataset, and demonstrates text-conditioned and object-guided motion synthesis on it.
Problem
Existing 3D human-object interaction datasets lack natural home settings that jointly capture body motion, dexterous hand manipulation, multiple objects, and articulated objects.
Approach
A hybrid capture system combining 70 synchronized RGB cameras with 3D ArUco cube markers for objects and IMU-based body suits and gloves, aligned via calibration protocols, to record parameterized HOI in a studio home apartment setting, plus generative motion synthesis experiments.
Key results
- Mixed-data MDM achieves R-Precision top3 0.73 and FID 0.85, outperforming ParaHome-only (FID 148.31) and HumanML3D-only (FID 74.66) training.
- Using 70 cameras yields marker detection ratio increasing linearly without saturation; surface-marker object tracking success is 0.76-0.93 versus 1.0 for the 3D cube marker.
- Hand alignment achieves an average fingertip position error of 11 mm over 86 finger touches.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train MDM on evenly mixed ParaHome and HumanML3D data instead of ParaHome alone | MDM motion synthesis · ParaHome + HumanML3D test set | FID: 148.31 -> 0.85 | Helped |
| Train MDM on evenly mixed ParaHome and HumanML3D data instead of HumanML3D alone | MDM motion synthesis · ParaHome + HumanML3D test set | R Precision (top3): 0.46 -> 0.73 | Helped |
| Use MDM rather than TEMOS on the evenly mixed dataset | Mixed-dataset motion synthesis · ParaHome + HumanML3D test set | FID: 7.16 -> 0.85 | Helped |
| Use MDM rather than T2M on the evenly mixed dataset | Mixed-dataset motion synthesis · ParaHome + HumanML3D test set | FID: 1.03 -> 0.85 | Helped |
| Use the 3D marker cube rather than 4 virtual surface markers | Object tracking · Object tracking: marker setup | Tracking success ratio: 0.76 -> 1.0 | Helped |
| Add the relative person-object spatial cue at the initial time (ablation) | Object-guided motion synthesis · Refrigerator | rc-joints MPE (cm): 1.05 -> 0.97 | Helped |
| Add the relative person-object spatial cue at the initial time (ablation) | Object-guided motion synthesis · Refrigerator | rc-wrists MPE (cm): 2.66 -> 2.09 | Helped |
| Add the relative person-object spatial cue at the initial time (ablation) | Object-guided motion synthesis · Refrigerator | Foot skating: 1.43 -> 1.74 | Hurt |
Surprising findings
- Training on the mixed dataset with T2M or TEMOS gives FID well below the real-data threshold, suggesting mixing is effective, though the paper does not analyze this in depth.
Limitations
- Relies on artificial markers; the authors plan a markerless approach.
- Captured in a single studio apartment setting; generalization to other rooms is untested.
- Synthetic RGB performance on pose estimation is lower than on existing benchmarks.
Experimental setup
Datasets: ParaHome HumanML3D AMASS ROPE FreiHAND HO3D
Benchmarks: HumanML3D
Metrics: R-Precision Multimodal Distance FID Diversity Multimodality MPE MOE Foot skating reprojection error tracking success ratio
Baselines: TEMOS T2M MDM
Builds on: MDM Text2Motion SMPL-X ControlNet HumanML3D
Closest papers by meaning
- InterAct: Advancing Large-Scale Versatile 3D Human-Object Interaction Generation (CVPR 2025 · similarity 0.90)
- HOI-M^3: Capture Multiple Humans and Objects Interaction within Contextual Environment (CVPR 2024 · similarity 0.87)
- CORE4D: A 4D Human-Object-Human Interaction Dataset for Collaborative Object REarrangement (CVPR 2025 · similarity 0.87)
- Scaling Up Dynamic Human-Scene Interaction Modeling (CVPR 2024 · similarity 0.87)
- DiffGrasp: Whole-Body Grasping Synthesis Guided by Object Motion Using a Diffusion Model (AAAI 2025 · similarity 0.86)
- Hand-Centric Motion Refinement for 3D Hand-Object Interaction via Hierarchical Spatial-Temporal Modeling (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.