Research map / 3D vision / 3D generation
Closely Interactive Human Reconstruction with Proxemics and Physics-Guided Adaption
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Reconstructs closely interacting two-person SMPL meshes from monocular video by refining an initial Gaussian distribution with a dual-branch VQ-VAE interaction prior and a proxemics/physics-guided dual-branch diffusion model.
Problem
Reconstructing closely interactive humans from monocular video is hard due to depth ambiguity and severe inter-person occlusion, and existing multi-person methods ignore close contact and proxemics.
Approach
Learn a dual-branch VQ-VAE codebook prior of two-person interactions, then treat diffusion as a distribution adaptor that starts from regressed initial Gaussian distributions and is guided each timestep by the prior's closest-code projection, a penetration (distance-field) loss, and 2D projection gradients.
Key results
- On Hi4D, the method reaches 63.1 MPJPE, 47.5 PA-MPJPE, and 81.4 interaction error, versus 82.4 MPJPE for GroupRec (best baseline).
- On 3DPW interactive sequences, MPJPE is 70.6 vs 72.9 for Human4D (best MPJPE), while interaction error is 100.3 vs 110.6 for GroupRec.
- Ablation on Hi4D: removing the discrete prior raises MPJPE from 63.1 to 67.6 and interaction error from 81.4 to 90.1.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use proxemics- and physics-guided distribution adaption | closely interactive human reconstruction · Hi4D | MPJPE: 96.8 -> 63.1 | Helped |
| use proxemics- and physics-guided distribution adaption | closely interactive human reconstruction · 3DPW interactive sequences | Interaction: 113.1 -> 100.3 | Helped |
| add distribution adaption to the initial prediction (ablation) | Ours · Hi4D | MPJPE: 73.7 -> 63.1 | Helped |
| use a dual-branch model instead of a single-branch model (ablation) | Ours · Hi4D | Interaction: 89.3 -> 81.4 | Helped |
| add the discrete interaction prior (ablation) | Ours · Hi4D | MPJPE: 67.6 -> 63.1 | Helped |
| add the discrete interaction prior (ablation) | Ours · Hi4D | Interaction: 90.1 -> 81.4 | Helped |
| add physical guidance (ablation) | Ours · Hi4D | Interaction: 86.1 -> 81.4 | Helped |
Surprising findings
- On 3DPW, the method does not achieve the best PA-MPJPE (51.4 vs 48.5 for BEV) or MPJPE vs Human4D's 72.9 is beaten, showing mixed joint-accuracy results outside the training domain.
Limitations
- Only supports two-person interactions; more people require more codebooks.
- Performance may deteriorate in outdoor/in-the-wild scenes since diffusion is trained on indoor data.
- Lacks an in-the-wild close-interaction dataset.
Experimental setup
Datasets: Human3.6M MPI-INF-3DHP COCO MPII InterHuman Hi4D CHI3D 3DPW
Benchmarks: Hi4D 3DPW CHI3D
Metrics: MPJPE PA-MPJPE MPVPE interaction error
Baselines: Human4D CLIFF BEV GroupRec BUDDI
Builds on: VQ-VAE diffusion models (DDPM) CLIFF PHALP BUDDI
Closest papers by meaning
- Reconstructing Close Human Interaction with Appearance and Proxemics Reasoning (CVPR 2025 · similarity 0.91)
- Generative Proxemics: A Prior for 3D Social Interaction from Images (CVPR 2024 · similarity 0.89)
- Template Free Reconstruction of Human-object Interaction with Procedural Interaction Generation (CVPR 2024 · similarity 0.87)
- Capturing Closely Interacted Two-Person Motions with Reaction Priors (CVPR 2024 · similarity 0.87)
- MultiPly: Reconstruction of Multiple People from Monocular Video in the Wild (CVPR 2024 · similarity 0.87)
- Real2Sim in HOI: Toward Physically Plausible HOI Reconstruction from Monocular Videos (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.