Research map / 3D vision / Face and shape
DLCA-Recon: Dynamic Loose Clothing Avatar Reconstruction from Monocular Videos
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DLCA-Recon, which reconstructs animatable avatars with loose clothing from monocular videos by initializing a canonical clothed mesh/SDF from ECON and using a physics-inspired dynamic deformation field plus refined skinning weights and pose.
Problem
Reconstructing dynamic humans in loose clothing from monocular video is hard because the clothing-to-body distance changes rapidly with free motion, and prior methods lack good geometric initialization and constraints, yielding discontinuous or incomplete surfaces.
Approach
Initialize the canonical space with an estimated clothed human (ECON) instead of SMPL, jointly optimize explicit mesh and implicit SDF, and model non-rigid clothing deformation with a dynamic deformation field based on force terms (traction from neighboring-frame joints, spring-like internal forces, environment variable), together with optimized skinning weights and pose refinement and delayed opt
Key results
- On DeepCap/DynaCap/self-captured sequences, the full model reduces normal MAE vs SelfRecon (e.g., LYZ 20.71 to 8.00 x10^-3) and raises mask IoU (FranziRed 0.735 to 0.912).
- Rendering PSNR on Antonia improves from 34.98 (SelfRecon) to 37.98, and on Magdalena from 36.07 to 39.77.
- Compared with SCARF on PeopleSnapshot, PSNR improves by roughly 6-12 dB across four subjects (e.g., female-3-casual 30.14 to 42.63).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DLCA-Recon instead of SelfRecon | Dynamic loose-clothing reconstruction · Antonia, Magdalena, FranziRed, LCJ, LYZ, and ZJ | Normal MAE*: 11.19 -> 6.14; 11.40 -> 7.69; 9.32 -> 5.29; 11.38 -> 7.63; 20.71 -> 8.00; 15.74 -> 9.46 | Helped |
| Use DLCA-Recon instead of SelfRecon | Dynamic loose-clothing reconstruction · Antonia, Magdalena, FranziRed, LCJ, LYZ, and ZJ | Mask IoU: 0.887 -> 0.904; 0.901 -> 0.912; 0.735 -> 0.912; 0.884 -> 0.910; 0.831 -> 0.919; 0.819 -> 0.906 | Helped |
| Add ECON initialization (ablation) | DLCA-Recon baseline · Lab (average over 3 sequences) | PSNR: 26.63 -> 32.03 | Helped |
| Add dynamic non-rigid deformation (ablation) | DLCA-Recon with initialization · Lab (average over 3 sequences) | PSNR: 32.03 -> 33.31 | Helped |
| Use the dynamic non-rigid field instead of the frame-index-only field (ablation) | DLCA-Recon · Lab (average over 3 sequences) | PSNR: 32.03 -> 33.81 | Helped |
| Add optimized skinning deformation fields and pose decoder to the other full-model components (ablation) | DLCA-Recon with initialization and dynamic non-rigid deformation · Lab (average over 3 sequences) | PSNR: 33.31 -> 33.81 | Helped |
| Replace PyMAF with SPIN (ablation) | DLCA-Recon · FranziRed | Normal MAE*: 5.29 -> 5.31 | No clear effect |
| Replace PyMAF with SPIN (ablation) | DLCA-Recon · FranziRed | PSNR: 34.09 -> 34.04 | No clear effect |
| Replace PyMAF with SPIN (ablation) | DLCA-Recon · FranziRed | IoU: 0.912 -> 0.912 | No clear effect |
| Replace PyMAF with SPIN (ablation) | DLCA-Recon · FranziRed | SSIM: 0.991 -> 0.991 | No clear effect |
| Remove delayed optimization (ablation) | DLCA-Recon · Training reconstruction | reconstruction quality | Hurt |
| Remove pose decoder (ablation) | DLCA-Recon · Pose refinement and reconstruction | reconstruction quality | Hurt |
Surprising findings
- Using SPIN instead of PyMAF for initial SMPL pose gives nearly identical results (FranziRed normal MAE 5.31 vs 5.29), suggesting low sensitivity to the pose estimator.
- The model has 73.45M parameters, far more than SelfRecon (3.79M), with inference time of 80.35 s versus 66.51 s for SelfRecon.
Limitations
- Limited by accuracy of ground-truth masks because surface rendering loss depends on them.
- Relies on predicted normal maps (PIFuHD) to refine geometric details.
- Geometry from 2D supervision is still inferior to 3D-supervised methods.
Experimental setup
Datasets: DeepCap DynaCap PeopleSnapshot ZJU-MoCap self-captured sequences (LCJ, LYZ, ZJ)
Benchmarks: DeepCap DynaCap PeopleSnapshot ZJU-MoCap
Metrics: normal MAE mask IoU PSNR SSIM
Baselines: SelfRecon HumanNeRF Vid2Avatar SCARF
Builds on: SelfRecon ECON IGR HumanNeRF IDR
Closest papers by meaning
- VS: Reconstructing Clothed 3D Human from Single Image via Vertex Shift (CVPR 2024 · similarity 0.90)
- D^3-Human: Dynamic Disentangled Digital Human from Monocular Video (CVPR 2025 · similarity 0.90)
- FreeCloth: Free-form Generation Enhances Challenging Clothed Human Modeling (CVPR 2025 · similarity 0.90)
- MPMAvatar: Learning 3D Gaussian Avatars with Accurate and Robust Physics-Based Dynamics (NeurIPS 2025 · similarity 0.88)
- FRESA: Feedforward Reconstruction of Personalized Skinned Avatars from Few Images (CVPR 2025 · similarity 0.88)
- CtrlAvatar: Controllable Avatars Generation via Disentangled Invertible Networks (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.