Research map / 3D vision / Human pose estimation
MUC: Mixture of Uncalibrated Cameras for Robust 3D Human Body Reconstruction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Fuses per-view SMPL-X estimates from a frozen-initialized SMPLer-X encoder across uncalibrated cameras using learned joint and surface reweighting networks.
Problem
Reconstructing 3D whole-body human meshes from multiple uncalibrated camera views, where calibration is cumbersome and self-occlusion and shape continuity are under-addressed.
Approach
Process each view with a pretrained single-view SMPLer-X encoder, then learn per-view, per-joint weights (Joint Reweighting Network) supervised by a distance-distribution KL loss to estimate self-occlusion, and fuse face/shape via a UV-map conditional U-Net (Surface Reweighting Network) conditioned on predicted camera position.
Key results
- On Human3.6M, MUC-base (SMPL-X) reaches 31.9 mm multi-view PA-MPJPE, vs. 33.0 for Yu and 28.2 for PaFF (SMPL), and MUC-base with translation reaches 27.1 mm PA-MPJPE.
- On RICH, MUC-base reduces multi-view PA-MPJPE from 42.1 (SMPLer-X-base) to 37.5 mm and PA-MPVPE from 37.1 to 33.5 mm.
- Adding cameras on RICH lowers body PA-MPJPE from 52.8 mm (1 camera) to 38.6 mm (4 cameras).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add JRN (ablation) | MUC · Human3.6M whole-body reconstruction | PA-MPJPE: 33.8 -> 32.5 | Helped |
| Add JRN (ablation) | MUC · RICH whole-body reconstruction | PA-MPJPE: 42.1 -> 39.4 | Helped |
| Add SRN after JRN (ablation) | MUC · Human3.6M whole-body reconstruction | PA-MPJPE: 32.5 -> 31.9 | Helped |
| Add SRN after JRN (ablation) | MUC · RICH whole-body reconstruction | PA-MPJPE: 39.4 -> 37.5 | Helped |
| Add SRN after JRN (ablation) | MUC · RICH face reconstruction | PA-MPVPE: 3.5 -> 3.3 | Helped |
| Increase camera count from 1 to 4 | MUC · RICH body reconstruction | PA-MPJPE: 52.8 -> 38.6 | Helped |
| Increase camera count from 1 to 4 | MUC · Human3.6M body reconstruction | PA-MPJPE: 47.0 -> 31.9 | Helped |
| Fuse multiple uncalibrated views with MUC instead of single-view SMPLer-X-base | Human3.6M reconstruction · Human3.6M multi-view reconstruction | PA-MPJPE: 45.1 -> 27.1 | Helped |
| Fuse multiple uncalibrated views with MUC instead of SMPLer-X-base | RICH reconstruction · RICH multi-view reconstruction | PA-MPJPE: 42.1 -> 37.5 | Helped |
Surprising findings
- MUC-base's multi-view PA-MPJPE on Human3.6M (31.9) is worse than PaFF's reported 28.2 mm, which is a SMPL-based method, so the SMPL-X model is not uniformly best on all metrics.
Limitations
- Currently limited to static images rather than video.
- Evaluation uses pseudo ground truth from NeuralAnnot and SMPLify-X rather than true SMPL-X annotations.
- Conversion to SMPL for comparison introduces vertex alignment errors.
Experimental setup
Datasets: Human3.6M RICH
Benchmarks: Human3.6M RICH
Metrics: PA-MPJPE PA-MPVPE MPJPE MPVPE
Baselines: SMPLer-X-base SMPLer-X-large PaFF ProHMR Yu Liang SPIN DecoMR PyMAF
Builds on: SMPLer-X SMPL-X PaFF Latent diffusion model (U-Net)
Closest papers by meaning
- Multiview Human Body Reconstruction from Uncalibrated Cameras (NeurIPS 2022 · similarity 0.92)
- Expressive Body Capture: 3D Hands, Face, and Body From a Single Image (CVPR 2019 · similarity 0.90)
- MultiPly: Reconstruction of Multiple People from Monocular Video in the Wild (CVPR 2024 · similarity 0.87)
- PSHuman: Photorealistic Single-image 3D Human Reconstruction using Cross-Scale Multiview Diffusion and Explicit Remeshing (CVPR 2025 · similarity 0.87)
- GeneMAN: Generalizable Single-Image 3D Human Reconstruction from Multi-Source Human Data (NeurIPS 2025 · similarity 0.86)
- ODHSR: Online Dense 3D Reconstruction of Humans and Scenes from Monocular Videos (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.