Research map / 3D vision / Human pose estimation
Reconstructing Close Human Interaction with Appearance and Proxemics Reasoning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a dual-branch optimization framework that fuses a diffusion-based proxemics prior with a 3D Gaussian splatting appearance branch to reconstruct close two-person interactions from in-the-wild monocular videos, and releases a 100-video WildCHI pseudo-ground-truth dataset.
Problem
Human pose estimation methods cannot recover plausible close two-person interactions from in-the-wild videos due to visual ambiguity, inter-person occlusion, depth ambiguity and mesh penetration.
Approach
Use human appearance (3D Gaussians splatted to the image and compared to the RGB frame) as a constraint jointly with a diffusion-learned proxemics/pose prior that is fine-tuned during optimization, rather than relying on 2D keypoints or masks alone.
Key results
- On Hi4D, the method reaches 59.1 MPJPE and 44.3 PA-MPJPE, versus 63.1 and 47.5 for CloseInt.
- On 3DPW interactive sequences, the method reaches 64.5 MPJPE versus 70.6 for CloseInt, while interaction error is slightly worse than CloseInt (96.4 vs 100.3 is better; the method is best on most metrics).
- Training CloseInt on WildCHI improves its Hi4D MPJPE from 63.1 to 61.4.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add appearance branch (3DGS constraint) to optimization (ablation) | dual-branch optimization on Hi4D · Hi4D | MPJPE: 60.68 -> 59.06 | Helped |
| add proxemics prior (diffusion model) (ablation) | dual-branch optimization on Hi4D · Hi4D | MPJPE: 61.52 -> 59.06 | Helped |
| add physics constraint (mesh penetration) (ablation) | dual-branch optimization on Hi4D · Hi4D | MPJPE: 57.01 -> 59.06 when removed | Mixed |
| add physics constraint (mesh penetration) (ablation) | dual-branch optimization on Hi4D · Hi4D | A-PD: 1.30 -> 0.81 | Helped |
| add appearance branch (ablation) | dual-branch optimization on Hi4D · Hi4D | Interaction error: 81.01 -> 80.18 | Helped |
| add proxemics prior (ablation) | dual-branch optimization on Hi4D · Hi4D | Interaction error: 87.13 -> 80.18 | Helped |
| add dual-branch optimization on top of initial prediction (ablation) | proxemics prior diffusion model · Hi4D | MPJPE: 65.05 -> 59.06 | Helped |
| train CloseInt on WildCHI pseudo ground-truth data | CloseInt · Hi4D | MPJPE: 63.1 -> 61.4 | Helped |
| train CloseInt on WildCHI pseudo ground-truth data | CloseInt · 3DPW | MPJPE: 70.6 -> 66.4 | Helped |
| dual-branch optimization framework vs. prior SOTA regression/optimization methods | close human interaction reconstruction · Hi4D and 3DPW | MPJPE: Hi4D 63.1 -> 59.1 (CloseInt); 3DPW 70.6 -> 64.5 (CloseInt) | Helped |
| add physics constraint (mesh penetration) (ablation) | dual-branch optimization on Hi4D · Hi4D | Interaction error: 78.50 -> 80.18 when removed (i.e., removal hurts ~1.7 with paper's ablation table reading) | Helped |
Surprising findings
- Removing the physics (penetration) constraint gives the lowest MPJPE (57.01) on Hi4D but worsens interaction error and A-PD, showing a trade-off.
- BUDDI, a dedicated close-interaction optimization method, performs worse than the single-person CLIFF and Human4D baselines on Hi4D.
Limitations
- Cannot reconstruct high-quality complete textures under changed lighting or partial observation.
- Only handles two-person interactions; no data for crowd proxemics priors.
- Input video needs some frames with little or no contact to constrain appearance.
Experimental setup
Datasets: Inter-X InterHuman Hi4D CHI3D 3DPW TikTok videos
Benchmarks: Hi4D 3DPW CHI3D
Metrics: MPJPE PA-MPJPE MPVPE interaction error A-PD
Baselines: Human4D CLIFF BEV GroupRec BUDDI CloseInt
Builds on: 3D Gaussian Splatting SMPL BUDDI CLIFF ViTPose
Closest papers by meaning
- Closely Interactive Human Reconstruction with Proxemics and Physics-Guided Adaption (CVPR 2024 · similarity 0.91)
- Generative Proxemics: A Prior for 3D Social Interaction from Images (CVPR 2024 · similarity 0.90)
- Capturing Closely Interacted Two-Person Motions with Reaction Priors (CVPR 2024 · similarity 0.90)
- Pose Priors from Language Models (CVPR 2025 · similarity 0.88)
- Template Free Reconstruction of Human-object Interaction with Procedural Interaction Generation (CVPR 2024 · similarity 0.88)
- InterAct: Advancing Large-Scale Versatile 3D Human-Object Interaction Generation (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.