Research map / 3D vision / Face and shape
VS: Reconstructing Clothed 3D Human from Single Image via Vertex Shift
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Vertex Shift (VS), a two-stage "stretch-refine" mesh deformation method that turns an estimated SMPL-X into a detailed clothed 3D human from a single image.
Problem
Single-image clothed 3D human reconstruction suffers from artifacts (holes, broken limbs, distortions) in implicit-function methods, while prior deformation methods cannot handle loose clothing.
Approach
Stretch the SMPL-X mesh with 2D shift fields inferred by cycle-consistent warping between body and clothing normal maps, then refine vertices with a mesh-convolutional GCN (DeformNet) driven by PIFuHD pixel-aligned features; only vertices move, never edges, so meshes stay watertight.
Key results
- On THuman2.0, VS reports Chamfer distance 0.628 cm vs 0.906 for ECON (ECON-EX) and 0.947 for ICON.
- On CAPE, VS reports Chamfer distance 0.621 cm vs 0.937 for ECON.
- In a 25-person perceptual study, VS was preferred in 58.1% (SHHQ) and 64.4% (VcgPeople) of cases.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add RefineVS to the StretchVS-only model (ablation) | VS · THuman 2.0 | Chamfer distance (cm): 0.979 -> 0.628 | Helped |
| add RefineVS to the StretchVS-only model (ablation) | VS · CAPE | cosine distance: 0.0404 -> 0.0262 | Helped |
| add RefineVS to the StretchVS-only model (ablation) | VS · RenderPeople | Chamfer distance (cm): 1.209 -> 0.976 | Helped |
| add StretchVS to the RefineVS-only model (ablation) | VS · THuman 2.0 | Chamfer distance (cm): 0.669 -> 0.628 | Helped |
| add StretchVS to the RefineVS-only model (ablation) | VS · CAPE | Chamfer distance (cm): 0.721 -> 0.621 | Helped |
| add StretchVS to the RefineVS-only model (ablation) | VS · RenderPeople | Chamfer distance (cm): 1.263 -> 0.976 | Helped |
| replace PIFuHD encoder with PIFu encoder (ablation) | VS · THuman 2.0 | Chamfer distance (cm): 0.628 -> 0.766 | Hurt |
| replace PIFuHD encoder with PaMIR encoder (ablation) | VS · THuman 2.0 | Chamfer distance (cm): 0.628 -> 0.838 | Hurt |
| use VS instead of ECON | single-image clothed 3D human reconstruction · THuman 2.0 | Chamfer distance (cm): 0.906 -> 0.628 | Helped |
| use VS instead of ECON | single-image clothed 3D human reconstruction · CAPE | point-to-surface distance (cm): + | Helped |
| use VS instead of ECON | single-image clothed 3D human reconstruction · RenderPeople | cosine distance: 0.0417 -> 0.0419 | No clear effect |
Surprising findings
- Using only RefineVS, the method degrades noticeably on RenderPeople, which contains out-of-distribution loose clothing, showing StretchVS is needed for loose garments.
- Swapping the feature encoder for PIFu or PaMIR changes quantitative results only modestly.
Limitations
- Fails on extreme clothing such as dresses on SMPL-X legs.
- Cannot distinguish body and clothing layers (layer confusion).
- Fails with large deformations along the Z-axis (e.g., belts drifting toward the camera).
Experimental setup
Datasets: THuman2.0
Benchmarks: THuman2.0 CAPE RenderPeople SHHQ VcgPeople
Metrics: Chamfer distance point-to-surface distance cosine distance L2 normal error
Baselines: PIFu PIFuHD PaMIR ICON ECON
Builds on: PIFuHD ECON MeshCNN PyMAF-X SMPL-X
Closest papers by meaning
- DLCA-Recon: Dynamic Loose Clothing Avatar Reconstruction from Monocular Videos (AAAI 2024 · similarity 0.90)
- Garment Recovery with Shape and Deformation Priors (CVPR 2024 · similarity 0.89)
- D^3-Human: Dynamic Disentangled Digital Human from Monocular Video (CVPR 2025 · similarity 0.89)
- Reconstruction of Manipulated Garment with Guided Deformation Prior (NeurIPS 2024 · similarity 0.88)
- FreeCloth: Free-form Generation Enhances Challenging Clothed Human Modeling (CVPR 2025 · similarity 0.88)
- MultiGO: Towards Multi-level Geometry Learning for Monocular 3D Textured Human Reconstruction (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.