Research map / 3D vision / Face and shape
GALA: Generating Animatable Layered Assets from a Single Scan
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
GALA decomposes a single-layer clothed 3D human scan into animatable, canonicalized layered assets (clothing/hair/accessories plus body) by inpainting occluded geometry and texture with a pose-guided SDS loss on a ControlNet-conditioned diffusion model.
Problem
Single-layer 3D human scans glue hair, clothing, and accessories into one mesh, so occluded layers are missing and meshes are not normalized in pose or shape, which blocks garment transfer and reanimation.
Approach
A pose-guided Score Distillation Sampling loss, using a pose-conditioned (OpenPose ControlNet) latent diffusion prior applied in canonical space plus reconstruction and segmentation losses in posed space, jointly decomposes, inpaints and canonicalizes each layer from one scan.
Key results
- On decomposition, GALA reaches CLIP text-image direction similarity 0.1117 and POR score 0.1144, versus 0.0621/0.4871 for Instruct-NeRF2NeRF and 0.0374/0.5583 for Vox-E.
- On canonicalization (CAPE), GALA attains 84.70% IoU and 0.821 cm Chamfer distance, beating nearest-neighbor (83.93%, 0.845) and KNN (83.93%, 0.846) skinning.
- Pose-guided SDS improves canonicalization over vanilla SDS and no canonical SDS: IoU 83.59% vs 82.89% vs 79.97%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use GALA instead of Instruct-NeRF2NeRF for object decomposition | 3D human scan decomposition · RenderPeople | CLIP text-image direction similarity: 0.0621 -> 0.1117 | Helped |
| use GALA instead of Instruct-NeRF2NeRF for object decomposition | 3D human scan decomposition · RenderPeople | pixel-wise object removal score: 0.4871 -> 0.1144 | Helped |
| use GALA instead of NN canonicalization | clothed human scan canonicalization · CAPE | IoU: 83.93% -> 84.70% | Helped |
| use GALA instead of Fast-SNARF trained with 200 scans | clothed human scan canonicalization · CAPE | IoU: 94.47% vs 84.70% | Hurt |
| add canonical-space SDS (ablation) | GALA canonicalization · CAPE | IoU: 79.97% -> 82.89% | Helped |
| add canonical-space SDS (ablation) | GALA canonicalization · CAPE | Chamfer distance: 1.384 -> 1.227 | Helped |
| use pose-guided SDS instead of vanilla SDS (ablation) | GALA canonicalization with canonical-space SDS · CAPE | IoU: 82.89% -> 83.59% | Helped |
| use pose-guided SDS instead of vanilla SDS (ablation) | GALA canonicalization with canonical-space SDS · CAPE | Chamfer distance: 1.227 -> 1.184 | Helped |
Surprising findings
- Fast-SNARF trained with 50 scans matches GALA's IoU (86.87%) from a single scan, showing that the baseline needs far more data to be competitive.
Limitations
- Relies on multi-view 2D segmentation and open-vocabulary segmentation to separate layers, so segmentation errors propagate.
- Canonicalization of a single scan is evaluated on a limited number of subjects (18 CAPE, 30 RenderPeople).
- Composition requires a penetration refinement step to resolve intersections.
Experimental setup
Datasets: RenderPeople CAPE
Benchmarks: RenderPeople (30 scans) CAPE (18 subjects)
Metrics: CLIP text-image direction similarity POR score IoU Chamfer distance
Baselines: Instruct-NeRF2NeRF Vox-E Nearest Neighbor skinning K-Nearest Neighbor skinning Fast-SNARF
Builds on: DreamFusion Fantasia3D DMTet ControlNet SMPL-X
Closest papers by meaning
- SiTH: Single-view Textured Human Reconstruction with Image-Conditioned Diffusion (CVPR 2024 · similarity 0.88)
- FreeCloth: Free-form Generation Enhances Challenging Clothed Human Modeling (CVPR 2025 · similarity 0.87)
- En3D: An Enhanced Generative Model for Sculpting 3D Humans from 2D Synthetic Data (CVPR 2024 · similarity 0.87)
- FRESA: Feedforward Reconstruction of Personalized Skinned Avatars from Few Images (CVPR 2025 · similarity 0.87)
- 3DGS-Avatar: Animatable Avatars via Deformable 3D Gaussian Splatting (CVPR 2024 · similarity 0.86)
- DeClotH: Decomposable 3D Cloth and Human Body Reconstruction from a Single Image (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.