Research map / 3D vision / Face and shape
DiffAvatar: Simulation-Ready Garment Optimization with Differentiable Simulation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
DiffAvatar co-optimizes body shape/pose, 2D garment pattern (via a control cage) and cloth material by differentiating through an XPBD simulator, recovering simulation-ready avatar garment assets from noisy 3D scans.
Problem
Creating simulation-ready garment assets (2D patterns, rest shape, material parameters) and body shape for clothed avatars from real scans is labor-intensive, and existing reconstruction methods only produce geometry unsuitable for physics simulation.
Approach
Integrates differentiable XPBD cloth simulation (DiffXPBD adjoint gradients) into the optimization loop so that garment 2D pattern (regularized by a mean-value-coordinate control cage with seam-length and curvature regularizers), material bending stiffness, and body shape/pose are jointly optimized to match a segmented scan.
Key results
- On the dress example, DiffAvatar reaches CD 1.311 vs. the GT scan against PoP 1.695 and PiFU-HD 1.930, and CD 1.688 vs. the artist mesh.
- Baselines PiFU-HD, Diffusion+NeuS and PoP all get a minimum mesh-quality score of 0, while DiffAvatar gets 0.143, indicating simulation-ready topology.
- Ablation: removing the control cage raises CD from 1.122 to 3.866 and produces inverted rest-shape triangles that break simulation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DiffAvatar instead of PoP | Dress reconstruction · Dress, against GT Scan | CD: 1.695 -> 1.311 | Helped |
| Use DiffAvatar instead of PoP | Dress reconstruction · Dress, against GT Scan | LPIPS: 0.140 -> 0.133 | Helped |
| Use DiffAvatar instead of PoP | Dress reconstruction · Dress, against GT Scan | SSIM: 0.831 -> 0.842 | Helped |
| Use DiffAvatar instead of PoP | Dress reconstruction · Dress, against GT Artist | CD: 1.866 -> 1.688 | Helped |
| Use DiffAvatar instead of PoP | Dress reconstruction · Dress, against GT Artist | LPIPS: 0.092 -> 0.085 | Helped |
| Use DiffAvatar instead of PoP | Dress reconstruction · Dress, against GT Artist | SSIM: 0.842 -> 0.893 | Helped |
| Add a control cage to pattern optimization (ablation) | DiffAvatar · Dress, against GT Scan | CD: 3.866 -> 1.122 | Helped |
| Add a control cage to pattern optimization (ablation) | DiffAvatar · Dress, against GT Scan | LPIPS: 0.168 -> 0.102 | Helped |
| Add a control cage to pattern optimization (ablation) | DiffAvatar · Dress, against GT Scan | SSIM: 0.792 -> 0.863 | Helped |
| Add seam length regularization (ablation) | DiffAvatar · Dress, against GT Scan | CD: 1.409 -> 1.122 | Helped |
| Add seam length regularization (ablation) | DiffAvatar · Dress, against GT Scan | LPIPS: 0.115 -> 0.102 | Helped |
| Add seam length regularization (ablation) | DiffAvatar · Dress, against GT Scan | SSIM: 0.843 -> 0.863 | Helped |
Surprising findings
- Optimizing directly over 2D pattern vertices without a control cage yields inverted non-physical triangles, causing simulator failure.
Limitations
- Optimization is slow (20-200 minutes per garment) and CPU-based.
- Cannot handle discrete changes in pattern piece count or mesh topology; starts from a category template.
- Assumes near quasi-equilibrium draping, so strong friction or dynamic effects and multi-layer occluded clothing are challenging.
Experimental setup
Datasets: 3dMD multi-view scans of 4 subjects artist-made garment meshes
Benchmarks: 3dMD scans of 4 subjects (dress example reported)
Metrics: Chamfer Distance LPIPS SSIM triangle conditioning mesh quality
Baselines: PiFU-HD Diffusion+NeuS PoP NeuralTailor
Builds on: DiffXPBD XPBD SMPL Mean Value Coordinates NeuralTailor
Closest papers by meaning
- SimAvatar: Simulation-Ready Avatars with Layered Hair and Clothing (CVPR 2025 · similarity 0.88)
- MPMAvatar: Learning 3D Gaussian Avatars with Accurate and Robust Physics-Based Dynamics (NeurIPS 2025 · similarity 0.88)
- DLCA-Recon: Dynamic Loose Clothing Avatar Reconstruction from Monocular Videos (AAAI 2024 · similarity 0.85)
- GarmentLab: A Unified Simulation and Benchmark for Garment Manipulation (NeurIPS 2024 · similarity 0.85)
- FreeCloth: Free-form Generation Enhances Challenging Clothed Human Modeling (CVPR 2025 · similarity 0.85)
- CtrlAvatar: Controllable Avatars Generation via Disentangled Invertible Networks (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.