Research map / 3D vision / 3D generation
Generative Proxemics: A Prior for 3D Social Interaction from Images
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BUDDI, a transformer-based diffusion model over SMPL-X parameters of two interacting people, used both as a generative model and as a score-distillation-style prior for single-image 3D reconstruction of close social interaction.
Problem
Reconstructing 3D poses and shapes of two people in close social interaction from a single image is hard due to mutual occlusion and scarce 3D training data with contact.
Approach
Learn a joint denoising diffusion prior over the SMPL-X parameters of two people using pseudo-ground-truth fits from contact-annotated images (FlickrCI3D) plus MoCap (CHI3D, Hi4D), then use one diffuse-denoise step per optimization iteration as a prior, removing the need for contact annotations at test time.
Key results
- On FlickrCI3D Signatures, BUDDI reduces joint PA-MPJPE to 66 mm from 106 mm for BEV, and gives the best PCC at each radius (e.g., 19 at r=5).
- On Hi4D, BUDDI achieves a per-person/joint PA-MPJPE of about 70/76 mm versus 78/84 mm for BEV, with the largest gains on hugging and kissing.
- In a forced-choice Mechanical Turk study, BUDDI samples were chosen over real training data 44.4% of the time (50% is chance), and over a VAE 60.17% of the time; BUDDI's FID is 1.6 versus 3.3 for the VAE.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use the BUDDI prior during optimization rather than BEV alone | BUDDI · FlickrCI3D Signatures | joint PA-MPJPE (mm): 106 -> 66 | Helped |
| use the BUDDI prior during optimization rather than the contact heuristic | BUDDI · FlickrCI3D Signatures | joint PA-MPJPE (mm): 68 -> 66 | Helped |
| use the BUDDI prior during optimization rather than BEV alone | BUDDI · Hi4D | joint PA-MPJPE (mm): 136 -> 98 | Helped |
| use the BUDDI prior during optimization rather than the contact heuristic | BUDDI · Hi4D | joint PA-MPJPE (mm): 121 -> 98 | Helped |
| use the BUDDI prior during optimization rather than BEV alone | BUDDI · CHI3D (pair s03) | joint PA-MPJPE (mm): 96 -> 68 | Helped |
| use the BUDDI prior during optimization rather than the contact heuristic | BUDDI · CHI3D (pair s03) | joint PA-MPJPE (mm): 105 -> 68 | Helped |
| remove optimization and use the conditional denoised output directly (ablation) | BUDDI · FlickrCI3D Signatures | joint PA-MPJPE (mm): 66 -> 92 | Hurt |
| replace diffusion with a direct single-step transformer prediction from BEV input (ablation) | BUDDI · FlickrCI3D Signatures | joint PA-MPJPE (mm): 66 -> 86 | Hurt |
Surprising findings
- The simple Contact Heuristic achieves lower reconstruction error than BUDDI on some joint metrics (e.g., PA-MPJPE 68 vs 66 is close; heuristic is best on handshake-like poses), showing that simple contact heuristics remain competitive for sparse contact.
- Single-person mesh regressor 4D Humans yields a joint PA-MPJPE of 344 mm on Flickr Fits, indicating poor two-person reconstruction.
Limitations
- BUDDI is not trained on hands or facial expressions because the datasets lack hand poses.
- Training data is small and partly pseudo-ground-truth from optimization fits with ground-truth contact labels, and MoCap data is limited to a few subject pairs.
- Evaluated only on two-person interactions; extension to more people is future work.
Experimental setup
Datasets: FlickrCI3D Signatures CHI3D Hi4D
Benchmarks: FlickrCI3D Signatures Hi4D CHI3D
Metrics: PA-MPJPE joint PA-MPJPE PCC FID forced-choice perceptual realism
Baselines: BEV Contact Heuristic Transformer (BUDDI single-step) VAE BUDDI (gen.) 4D Humans
Builds on: denoising diffusion probabilistic models SMPLify VPoser Score distillation sampling
Closest papers by meaning
- Reconstructing Close Human Interaction with Appearance and Proxemics Reasoning (CVPR 2025 · similarity 0.90)
- Closely Interactive Human Reconstruction with Proxemics and Physics-Guided Adaption (CVPR 2024 · similarity 0.89)
- Template Free Reconstruction of Human-object Interaction with Procedural Interaction Generation (CVPR 2024 · similarity 0.89)
- ChatHuman: Chatting about 3D Humans with Tools (CVPR 2025 · similarity 0.86)
- HOI-M^3: Capture Multiple Humans and Objects Interaction within Contextual Environment (CVPR 2024 · similarity 0.86)
- Pose Priors from Language Models (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.