Research map / 3D vision / Human pose estimation
Capturing Closely Interacted Two-Person Motions with Reaction Priors
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes reaction priors (motion VAE plus invertible neural network) and a query-based decoder-only Transformer pose estimator to capture closely interacted two-person motions from monocular video, and releases the Dual-Human dataset (~2k sequences).
Problem
Capturing closely interacted two-person motion from monocular video is hard because frequent inter-human occlusions make single-person and multi-person pose estimators unreliable; existing interacted-motion datasets are small or lack accurate, contact-annotated close interactions.
Approach
Learns reaction priors as a VAE latent space plus an invertible neural network that bi-directionally maps one person's motion distribution to the other's, used both as a training regularizer (KL prior loss) and test-time optimization guidance, combined with an interaction-aware self-attention query-based Transformer pose estimator.
Key results
- On Dual-Human, the method reaches 63.4 mm MPJPE and 112.1 mm translation error, versus 67.8 mm and 120.9 mm for TRACE (best baseline).
- On Hi4D, MPJPE improves from 75.0 mm to... (best baseline CLIFF 77.9 mm MPJPE; ours 75.0 mm) and on CHI3D it reaches 71.9 mm versus 73.1 mm for 4D-Humans.
- Reaction priors outperform BUDDI proxemic priors on a 10% Dual-Human test subset (MPJPE 64.1 vs 67.3 mm).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use the proposed framework with reaction priors | two-person monocular motion capture · Hi4D | MPJPE (mm): 77.9 -> 75.0 | Helped |
| use the proposed framework with reaction priors | two-person monocular motion capture · CHI3D | MPJPE (mm): 73.1 -> 71.9 | Helped |
| use the proposed framework with reaction priors | two-person monocular motion capture · Dual-Human | MPJPE (mm): 66.9 -> 63.4 | Helped |
| use reaction priors instead of BUDDI proxemics priors | the same pose estimator on Dual-Human test subset (10%) · Dual-Human | MPJPE (mm): 67.3 -> 64.1 | Helped |
| remove motion VAE (ablation) | reaction generation · Dual-Human | MPJPE (mm): 63.4 -> 64.4 | Hurt |
| replace INN with MLPs (ablation) | reaction generation · Dual-Human | MPJPE (mm): 63.4 -> 66.2 | Hurt |
| remove noise augmentation (ablation) | reaction generation · Dual-Human | MPJPE (mm): 63.4 -> 77.4 | Hurt |
| remove interaction-aware self-attention (ablation) | pose estimator · Dual-Human | MPJPE (mm): 63.4 -> 70.3 | Hurt |
| remove reaction priors (ablation) | MoCap framework · Dual-Human | MPJPE (mm): 63.4 -> 67.9 | Hurt |
| remove temporal network (ablation) | MoCap framework · Dual-Human | MPJPE (mm): 63.4 -> 63.6 | No clear effect |
Limitations
- Reaction priors may not generalize to interactions far from the dataset's distribution.
- Large prior weights can produce false interaction motions for non-interacting inputs.
- Assumes exactly two people in the scene; more people require extra processing.
Experimental setup
Datasets: Dual-Human Hi4D CHI3D
Benchmarks: Hi4D CHI3D Dual-Human
Metrics: MPJPE PA-MPJPE Translation error
Baselines: CLIFF 4D-Humans BEV TRACE BUDDI
Builds on: VPoser HuMoR ProHMR 4D-Humans SmoothNet
Closest papers by meaning
- Reconstructing Close Human Interaction with Appearance and Proxemics Reasoning (CVPR 2025 · similarity 0.90)
- Pose Priors from Language Models (CVPR 2025 · similarity 0.89)
- Motions as Queries: One-Stage Multi-Person Holistic Human Motion Capture (CVPR 2025 · similarity 0.89)
- InterAct: Advancing Large-Scale Versatile 3D Human-Object Interaction Generation (CVPR 2025 · similarity 0.87)
- Closely Interactive Human Reconstruction with Proxemics and Physics-Guided Adaption (CVPR 2024 · similarity 0.87)
- ContactField: Implicit Field Representation for Multi-Person Interaction Geometry (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.