Research map / 3D vision / Face and shape
SCULPT: Shape-Conditioned Unpaired Learning of Pose-dependent Clothed and Textured Human Meshes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Generative model (SCULPT) of pose-dependent clothed, textured SMPL-based meshes, trained unpaired: geometry from CAPE 3D scans and texture from 2D fashion images via adversarial losses, with geometry-conditioned texture generation.
Problem
Generating explicit 3D clothed and textured human meshes is hard because textured 3D human scan datasets are small and scarce, while paired geometry-appearance data is unavailable.
Approach
Learn pose-dependent clothing geometry as UV displacement maps on SMPL from 3D scans, then train a texture StyleGAN unsupervised on 2D fashion images conditioned on intermediate features of the geometry generator and on CLIP/BLIP-derived attribute labels for clothing type and color.
Key results
- SCULPT achieves FID 9.85 vs. EG3D 7.38 and EVA3D 44.11 on rendered images.
- Ablation: removing geometry conditioning raises FID from 9.85 to 28.2 (full model with both global and 64x64 patch discriminators).
- SCULPT texture quality is comparable to EG3D while its geometry is reported as higher quality, though geometry is not quantitatively evaluated.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add geometry-generator feature conditioning to the texture generator (ablation) | SCULPT · SCULPT dataset | FID: 28.2 -> 9.85 | Helped |
| combine global and local discriminators instead of using only the global discriminator (ablation) | SCULPT · SCULPT dataset | FID: 31.6 -> 9.85 | Helped |
| use a 64×64 local discriminator instead of a 32×32 local discriminator, with no global discriminator (ablation) | SCULPT · SCULPT dataset | FID: 24.5 -> 16.1 | Helped |
| use a 64×64 rather than a 32×32 local discriminator alongside the global discriminator (ablation) | SCULPT · SCULPT dataset | FID: 19.2 -> 9.85 | Helped |
| use both global and 64×64 local discriminators instead of only the global discriminator (ablation) | SCULPT · SCULPT dataset | FID: 31.6 -> 9.85 | Helped |
| use both global and 64×64 local discriminators instead of only a 64×64 local discriminator (ablation) | SCULPT · SCULPT dataset | FID: 16.1 -> 9.85 | Helped |
Surprising findings
- EG3D yields better FID than SCULPT (7.38 vs 9.85) despite the paper's claim of comparable rendering quality.
Limitations
- Poor texture quality on the back of the body and for out-of-distribution poses.
- Topological limits: cannot model loose clothing such as skirts or dresses with fixed SMPL topology.
- Dataset biases toward frontal views, male subjects, and typical fashion poses.
Experimental setup
Datasets: CAPE Zalando fashion images (16362)
Benchmarks: SCULPT dataset
Metrics: FID KID precision/recall
Baselines: EG3D EVA3D GET3D StylePeople
Builds on: StyleGAN3 EG3D SMPL CAPE StyleGAN-Ada
Closest papers by meaning
- Learning to Dress 3D People in Generative Clothing (CVPR 2020 · similarity 0.92)
- MultiGO: Towards Multi-level Geometry Learning for Monocular 3D Textured Human Reconstruction (CVPR 2025 · similarity 0.89)
- SiTH: Single-view Textured Human Reconstruction with Image-Conditioned Diffusion (CVPR 2024 · similarity 0.88)
- En3D: An Enhanced Generative Model for Sculpting 3D Humans from 2D Synthetic Data (CVPR 2024 · similarity 0.88)
- Design2Cloth: 3D Cloth Generation from 2D Masks (CVPR 2024 · similarity 0.88)
- FreeCloth: Free-form Generation Enhances Challenging Clothed Human Modeling (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.