Research map / 3D vision / Gaussian splatting
Disco4D: Disentangled 4D Human Generation and Animation from a Single Image
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Disco4D generates animatable 4D clothed human Gaussian avatars from a single image by disentangling clothing Gaussians (identity-encoded, fitted over the SMPL-X body Gaussians) from the fixed body.
Problem
Single-image 3D/4D clothed human generation usually yields single-layer, non-animatable meshes where clothing, hair, accessories and body are merged, limiting re-animation, editing and asset extraction; layered methods need multi-view or video inputs.
Approach
Bind fixed SMPL-X body Gaussians and fit separate clothing Gaussians outside the body, using a visual hull for initialization, learnable identity encodings tied to SegFormer categories for separation, and SDS from diffusion models to inpaint unseen regions; animate by driving SMPL-X plus a learned deformation network for clothing.
Key results
- On SynBody, Disco4D reaches CLIP All 0.851 versus 0.807 for LGM and 0.751 for DreamGaussian, with PSNR 15.69 (SHERF 15.19).
- On 4D-Dress, Disco4D (reposed)+learned deformations achieves CLIP All 0.900 and PSNR 25.46, versus 0.784 and 20.54 for DreamGaussian4D.
- In a 43-person user study, Disco4D scored 3.142 for image consistency and 3.037 for overall quality, above LGM (2.338, 2.017) and DreamGaussian (2.017, 1.852).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Disco4D instead of LGM | single-image 3D human generation · SynBody | CLIP similarity (All): 0.807 -> 0.851 | Helped |
| use Disco4D instead of SHERF | single-image 3D human generation · SynBody novel view | PSNR: 15.189 -> 15.691 | Helped |
| use Disco4D instead of LGM | single-image 3D human generation · CloSe | CLIP similarity (All): 0.829 -> 0.856 | Helped |
| use Disco4D instead of LGM | single-image 3D human generation · CloSe novel view | PSNR: 20.50 -> 20.10 | Hurt |
| use Disco4D instead of SHERF | single-image 3D human generation · CloSe novel pose | PSNR: 15.54 -> 17.96 | Helped |
| use Disco4D instead of LGM | image-to-3D generation · SHHQ user study | Image consistency rating: 2.338 -> 3.142 | Helped |
| use Disco4D instead of LGM | image-to-3D generation · SHHQ user study | Overall quality rating: 2.017 -> 3.037 | Helped |
| add learned clothing deformations to the reposed model (ablation) | Disco4D (reposed) · 4D-Dress | PSNR: 23.94 -> 25.46 | Helped |
| add learned clothing deformations to the reposed model (ablation) | Disco4D (reposed) · 4D-Dress | CLIP similarity (All): 0.853 -> 0.900 | Helped |
| add learned clothing deformations to the reposed model (ablation) | Disco4D (reposed) · 4D-Dress | CLIP similarity (Assets): 0.774 -> 0.865 | Helped |
| add learned clothing deformations to the reposed model (ablation) | Disco4D (reposed) · 4D-Dress | SSIM: 0.95 -> 0.96 | Helped |
| add learned clothing deformations to the reposed model (ablation) | Disco4D (reposed) · 4D-Dress | LPIPS: 0.049 -> 0.035 | Helped |
Surprising findings
- Disco4D performs worse than LGM on novel views, which the authors attribute to canonical-space optimization trading view-specific detail for pose generalization.
- Hull-based initialization of clothing Gaussians outperforms random and SMPL-X surface initialization in the ablation.
Limitations
- Relies on robust, pixel-aligned SMPL-X estimation and fails when the visual hull initialization is poor.
- Mesh extraction from clothing Gaussians loses fine-grained geometric detail.
- Does not yet model multi-layered clothing or reconstruct occluded clothing.
Experimental setup
Datasets: SynBody CloSe 4D-Dress SHHQ
Benchmarks: SynBody CloSe 4D-Dress
Metrics: CLIP similarity PSNR SSIM LPIPS user study rating (1-5)
Baselines: DreamGaussian LGM SHERF DreamGaussian4D MonoHuman GART GaussianAvatar Animate-Anyone Magic-Animate CHAMP
Builds on: DreamGaussian LGM DreamGaussian4D SuGaR SplattingAvatar
Closest papers by meaning
- Human Gaussian Splatting: Real-time Rendering of Animatable Avatars (CVPR 2024 · similarity 0.87)
- HUGS: Human Gaussian Splats (CVPR 2024 · similarity 0.87)
- 4D-DRESS: A 4D Dataset of Real-World Human Clothing With Semantic Annotations (CVPR 2024 · similarity 0.87)
- HumanSplat: Generalizable Single-Image Human Gaussian Splatting with Structure Priors (NeurIPS 2024 · similarity 0.87)
- GauHuman: Articulated Gaussian Splatting from Monocular Human Videos (CVPR 2024 · similarity 0.86)
- Align Your Gaussians: Text-to-4D with Dynamic 3D Gaussians and Composed Diffusion Models (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.