Research map / 3D vision / Face and shape
Garment Recovery with Shape and Deformation Priors
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Recovers realistic, directly usable 3D loose-garment meshes from single in-the-wild images by combining an ISP sewing-pattern shape prior with a learned deformation model and two-stage image-based fitting.
Problem
Recovering accurate, simulation-ready 3D garment meshes of loose-fitting clothing from single real-world images, where existing methods are limited to tight clothing or produce watertight/point-cloud outputs.
Approach
Extends the Implicit Sewing Patterns (ISP) garment model with a deformation network that predicts occupancy and position maps conditioned on normal images and SMPL body features, learned from synthetic CLOTH3D data, then fits its pretrained parameters to real images with silhouette, normal, and physics losses followed by per-vertex refinement.
Key results
- On synthetic CLOTH3D test garments, the method reaches CD 0.44e-3 (shirt), 2.87e-3 (skirt), 0.28e-3 (trousers) and IoU 0.939-0.946, outperforming SMPLicit, DrapeNet, and ISP.
- Vertex refinement and deformation-model fine-tuning both improve results in ablation; skirt CD drops from 3.58e-3 to 2.87e-3 with both steps on estimated normals.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add deformation-model fine-tuning and vertex-position optimization (ablation) | Garment Recovery with Shape and Deformation Priors · CLOTH3D test skirts, GT normals | CD (×103): 2.04 -> 1.67 | Helped |
| Add deformation-model fine-tuning and vertex-position optimization (ablation) | Garment Recovery with Shape and Deformation Priors · CLOTH3D test skirts, estimated normals | CD (×103): 3.58 -> 2.87 | Helped |
| Add deformation-model fine-tuning and vertex-position optimization (ablation) | Garment Recovery with Shape and Deformation Priors · CLOTH3D test skirts, GT normals | IoU: 0.848 -> 0.953 | Helped |
| Add deformation-model fine-tuning and vertex-position optimization (ablation) | Garment Recovery with Shape and Deformation Priors · CLOTH3D test skirts, estimated normals | IoU: 0.751 -> 0.940 | Helped |
| Add deformation-model fine-tuning (ablation) | Garment Recovery with Shape and Deformation Priors · CLOTH3D test skirts, GT normals | CD (×103): 1.80 -> 1.67 | Helped |
| Add deformation-model fine-tuning (ablation) | Garment Recovery with Shape and Deformation Priors · CLOTH3D test skirts, estimated normals | CD (×103): 3.39 -> 2.87 | Helped |
| Add deformation-model fine-tuning (ablation) | Garment Recovery with Shape and Deformation Priors · CLOTH3D test skirts, GT normals | IoU: 0.947 -> 0.953 | Helped |
| Add deformation-model fine-tuning (ablation) | Garment Recovery with Shape and Deformation Priors · CLOTH3D test skirts, estimated normals | IoU: 0.904 -> 0.940 | Helped |
| Add vertex-position optimization (ablation) | Garment Recovery with Shape and Deformation Priors · CLOTH3D test skirts, GT normals | CD (×103): 1.75 -> 1.67 | Helped |
| Add vertex-position optimization (ablation) | Garment Recovery with Shape and Deformation Priors · CLOTH3D test skirts, estimated normals | CD (×103): 2.93 -> 2.87 | Helped |
| Add vertex-position optimization (ablation) | Garment Recovery with Shape and Deformation Priors · CLOTH3D test skirts, GT normals | IoU: 0.941 -> 0.953 | Helped |
| Add vertex-position optimization (ablation) | Garment Recovery with Shape and Deformation Priors · CLOTH3D test skirts, estimated normals | IoU: 0.923 -> 0.940 | Helped |
Surprising findings
- Using ground-truth normals instead of estimated ones only modestly improves accuracy, indicating robustness to normal-estimation errors.
- Directly optimizing vertices without fine-tuning the deformation model is less effective and yields implausible surfaces.
Limitations
- Evaluation on real images is qualitative only, since no public real garment dataset with ground truth is available.
- Relies on off-the-shelf normal estimation and SMPL body estimation, and on synthetic training data with limited material and motion diversity.
- Handles only shirts, skirts, and trousers in the quantitative study.
Experimental setup
Datasets: CLOTH3D
Benchmarks: CLOTH3D unseen garments (30 shirts, 30 skirts, 30 trousers)
Metrics: Chamfer Distance IoU
Baselines: BCNet SMPLicit ClothWild DrapeNet ISP
Builds on: Implicit Sewing Patterns (ISP) SMPL ECON CLOTH3D
Closest papers by meaning
- Reconstruction of Manipulated Garment with Guided Deformation Prior (NeurIPS 2024 · similarity 0.92)
- VS: Reconstructing Clothed 3D Human from Single Image via Vertex Shift (CVPR 2024 · similarity 0.89)
- FreeCloth: Free-form Generation Enhances Challenging Clothed Human Modeling (CVPR 2025 · similarity 0.88)
- DressWild: Feed-Forward Pose-Agnostic Garment Sewing Pattern Generation from In-the-Wild Images (NeurIPS 2026 · similarity 0.88)
- Learning 3D Garment Animation from Trajectories of A Piece of Cloth (NeurIPS 2024 · similarity 0.87)
- GaPT-DAR: Category-level Garments Pose Tracking via Integrated 2D Deformation and 3D Reconstruction (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.