Research map / 3D vision / Human pose estimation
MultiGO: Towards Multi-level Geometry Learning for Monocular 3D Textured Human Reconstruction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MultiGO, a multi-level geometry learning framework for monocular 3D textured human reconstruction that adds skeleton-level Fourier-projection features, joint-level depth augmentation, and diffusion-style wrinkle refinement on top of an object Gaussian reconstruction model.
Problem
Monocular 3D clothed human reconstruction suffers from single-view ambiguity; existing SMPL(-X)-prior or generative methods capture only general body geometry, yielding inaccurate skeletons, wrong joint positions, and unclear cloth wrinkles.
Approach
Projects 3D Fourier features of the SMPL-X point cloud into 2D from three views and fuses them with image features in a Gaussian reconstruction UNet (SLE); randomly perturbs SMPL-X joint parameters during training (JLA) for robustness to depth-prior errors; and treats the coarse normal map as a noised latent to refine with a fine-tuned latent diffusion UNet (WLR).
Key results
- On CustomHuman, MultiGO reaches CD 1.620/1.782 cm, NC 0.850, f-score 42.425, improving over the second-best SiTH (CD 1.800/2.188, NC 0.816, f-score 36.148).
- On THuman3.0, MultiGO achieves NC 0.834 and f-score 46.091 versus 0.787 and 36.230 for SiTH.
- Texture PSNR on CustomHuman rises to 22.347 (front) versus 18.417 for the best prior SiTH in Table 2 comparison.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the SLE projection of 3D Fourier features into 2D (ablation) | MultiGO · CustomHuman and THuman3.0 | f-score: 31.624 -> 42.425; 35.086 -> 46.091 | Helped |
| add the SMPL-X shape prior (ablation) | MultiGO · CustomHuman and THuman3.0 | f-score: 33.295 -> 42.425; 37.685 -> 46.091 | Helped |
| use three projection views instead of two (ablation) | MultiGO SLE · CustomHuman and THuman3.0 | f-score: 38.651 -> 42.425; 43.255 -> 46.091 | Helped |
| use three projection views instead of one (ablation) | MultiGO SLE · CustomHuman and THuman3.0 | f-score: 36.835 -> 42.425; 42.802 -> 46.091 | Helped |
| add JLA perturbations at α=0.25 instead of using α=0.10 (ablation) | MultiGO · CustomHuman and THuman3.0 | f-score: 41.418 -> 42.425; 44.008 -> 46.091 | Helped |
| add JLA perturbations at α=0.25 (ablation) | MultiGO · CustomHuman and THuman3.0 | f-score: 40.429 -> 42.425; 43.166 -> 46.091 | Helped |
| add WLR (ablation) | MultiGO · CustomHuman and THuman3.0 | f-score: 42.081 -> 42.425; 45.525 -> 46.091 | Helped |
| add SLE (ablation) | MultiGO · CustomHuman and THuman3.0 | LPIPS F/B: 0.0497/0.0750 -> 0.0414/0.0643; 0.0575/0.0676 -> 0.0457/0.0616 | Helped |
| add JLA (ablation) | MultiGO · CustomHuman and THuman3.0 | LPIPS F/B: 0.0465/0.0666 -> 0.0414/0.0643; 0.0541/0.0657 -> 0.0457/0.0616 | Helped |
Surprising findings
- Directly encoding 3D SMPL-X Fourier features with convolutions (without 2D projection) degrades performance below the no-prior baseline in some metrics.
Limitations
- Evaluation relies on estimated SMPL/SMPL-X priors rather than ground truth, and the wrinkle module's quantitative gain is small.
- Depends on a limited amount of human scan data for training, with the diffusion refinement fine-tuned on limited human data.
Experimental setup
Datasets: THuman2.0 CustomHuman THuman3.0
Benchmarks: CustomHuman THuman3.0
Metrics: Chamfer distance normal consistency f-score LPIPS SSIM PSNR
Baselines: PIFu GTA ICON ECON VS HiLo SiFU SiTH
Builds on: Gaussian reconstruction model (Szymanowicz et al.) LRM SMPL-X latent diffusion model SiTH
Closest papers by meaning
- SCULPT: Shape-Conditioned Unpaired Learning of Pose-dependent Clothed and Textured Human Meshes (CVPR 2024 · similarity 0.89)
- PSHuman: Photorealistic Single-image 3D Human Reconstruction using Cross-Scale Multiview Diffusion and Explicit Remeshing (CVPR 2025 · similarity 0.88)
- SiTH: Single-view Textured Human Reconstruction with Image-Conditioned Diffusion (CVPR 2024 · similarity 0.88)
- Diffusion-FOF: Single-View Clothed Human Reconstruction via Diffusion-Based Fourier Occupancy Field (CVPR 2024 · similarity 0.88)
- VS: Reconstructing Clothed 3D Human from Single Image via Vertex Shift (CVPR 2024 · similarity 0.88)
- SIFU: Side-view Conditioned Implicit Function for Real-world Usable Clothed Human Reconstruction (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.