Research map / 3D vision / Human pose estimation
UniCon3R: Unified Contact-aware 4D Human-Scene Reconstruction from Monocular Video
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes UniCon3R, a feed-forward online 4D human-scene reconstruction model that infers dense body-scene contact as a prompt and feeds it back into the human mesh regression via latent refinement, reducing floating/penetration artifacts.
Problem
Feed-forward unified human-scene reconstruction from monocular video yields physically implausible results (floating feet, penetration) because human-scene interaction is handled only implicitly.
Approach
A scene-aware contact prompt (fusing human token, current-frame and memory scene context, local metric geometry, and temporal momentum) predicts dense per-vertex SMPL contact, and a contact-guided latent residual feeds the contact representation back into the human latent before SMPL-X regression, so contact acts as a corrective signal rather than an auxiliary output.
Key results
- On RICH, UniCon3R reduces WA-MPJPE from 97.5 (Human3R dagger) to 81.5 mm and Penetrate by about 75% relative to Human3R released checkpoint.
- On RICH contact estimation, UniCon3R achieves F1 0.71 and geodesic error 14.98 vs DECO F1 0.70 and 17.92.
- Runs at 2.40 FPS on a single A5000, matching Human3R's 2.41 FPS, while adding negligible overhead.
Surprising findings
- Predicting contact as an auxiliary parallel output (Parallel Readout) slightly worsens global motion error relative to Human3R dagger, showing contact supervision alone is not sufficient.
- Human3R-derived contact via distance thresholding gets very low F1 (0.08), indicating implicit models do not capture contact.
Limitations
- Contact is represented as binary per-vertex labels, not richer physical quantities such as forces or friction.
- Quality of contact depends on reconstructed scene geometry; scene errors propagate.
- Rigid support geometry assumption; deformable or fine-grained object interactions not modeled.
Experimental setup
Datasets: RICH BEDLAM
Benchmarks: RICH EMDB-2 SLOPER4D 3DPW
Metrics: WA-MPJPE W-MPJPE RTE PA-MPJPE PVE MPJPE Penetrate Float Pen. Max Collision Ratio
Baselines: Human3R UniSH JOSH3R Parallel Readout DECO InteractVLM JOSH SLAHMR TRAM WHAM GVHMR
Builds on: CUT3R Human3R Multi-HMR DECO BSTRO
Closest papers by meaning
- Human3R: Everyone Everywhere All at Once (ICLR 2026 · similarity 0.91)
- ODHSR: Online Dense 3D Reconstruction of Humans and Scenes from Monocular Videos (CVPR 2025 · similarity 0.89)
- Reconstructing People, Places, and Cameras (CVPR 2025 · similarity 0.88)
- Synergistic Global-space Camera and Human Reconstruction from Videos (CVPR 2024 · similarity 0.88)
- Real2Sim in HOI: Toward Physically Plausible HOI Reconstruction from Monocular Videos (NeurIPS 2026 · similarity 0.87)
- 4D3R: Motion-Aware Neural Reconstruction and Rendering of Dynamic Scenes from Monocular Videos (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.