Research map / 3D vision / Face and shape
SIFU: Side-view Conditioned Implicit Function for Real-world Usable Clothed Human Reconstruction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SIFU, which reconstructs clothed humans from a single image using a side-view decoupling transformer (SMPL-X normals as cross-attention queries) plus diffusion-based 3D consistent texture refinement.
Problem
Single-image 3D clothed human reconstruction struggles with complex poses, loose clothing, and texture prediction for unseen views, due to weak prior guidance when lifting 2D features to 3D.
Approach
Use rendered side-view SMPL-X normals as queries in a cross-attention transformer over image features to decouple side-view features, fuse them with a hybrid prior query strategy in an implicit function, then refine textures with a text-to-image diffusion model under multi-view consistent editing.
Key results
- Achieves Chamfer 0.5961 cm and P2S 0.6058 cm on THuman2.0, the best among compared methods (GTA 0.7329 / 0.7297).
- On CAPE-NFP, Chamfer 0.7725 vs GTA 0.8508 and ICON 0.8846 with ground-truth SMPL-X.
- Texture PSNR improves 22.2% over PIFu; under SMPL-X noise, Chamfer 0.9937 vs GTA 1.0473.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use SIFU’s side-view conditioned implicit function and texture refinement instead of GTA | SIFU · THuman2.0 | Chamfer: 0.7329 -> 0.5961 | Helped |
| use SIFU instead of GTA with noisy SMPL-X estimates | SIFU · THuman2.0 (noisy SMPL-X) | Chamfer: 1.0473 -> 0.9937 | Helped |
| add cross-attention instead of omitting it (ablation) | SIFU · CAPE-NFP | Chamfer: 0.9846 -> 0.7725 | Helped |
| use cross-attention instead of learnable embeddings (ablation) | SIFU · CAPE-NFP | Chamfer: 0.9860 -> 0.7725 | Helped |
| use the transformer instead of a convolution network (ablation) | SIFU · CAPE-NFP | Chamfer: 0.8699 -> 0.7725 | Helped |
| add left, back, and right feature planes to the front plane (ablation) | SIFU · CAPE-NFP | Chamfer: 1.1165 -> 0.7725 | Helped |
| add left and right feature planes to the front and back planes (ablation) | SIFU · CAPE-NFP | Chamfer: 0.9929 -> 0.7725 | Helped |
| add the left feature plane (ablation) | SIFU · CAPE-NFP | Chamfer: 0.7941 -> 0.7725 | Helped |
| add the right feature plane (ablation) | SIFU · CAPE-NFP | Chamfer: 0.8058 -> 0.7725 | Helped |
| use hybrid prior fusion instead of pixel-aligned querying (ablation) | SIFU · CAPE-NFP | Chamfer: 0.8111 -> 0.7725 | Helped |
Surprising findings
- Adding only left or right side-view planes gives most of the accuracy gain; adding all four planes yields only about 0.03 cm further reduction.
Limitations
- Depends on accurate SMPL-X estimates; robustness only tested with synthetic noise on ground-truth SMPL-X.
- Trained only on THuman2.0 (490 scans), limiting data diversity for texture prediction.
Experimental setup
Datasets: THuman2.0 CAPE
Benchmarks: THuman2.0 CAPE-FP CAPE-NFP
Metrics: Chamfer distance P2S L2 normal error PSNR
Baselines: PIFu PIFuHD PaMIR ICON ECON D-IF GTA TEXTure DreamGaussian
Builds on: PIFu GTA ECON TEXTure DreamGaussian
Closest papers by meaning
- Diffusion-FOF: Single-View Clothed Human Reconstruction via Diffusion-Based Fourier Occupancy Field (CVPR 2024 · similarity 0.88)
- MultiGO: Towards Multi-level Geometry Learning for Monocular 3D Textured Human Reconstruction (CVPR 2025 · similarity 0.88)
- SiTH: Single-view Textured Human Reconstruction with Image-Conditioned Diffusion (CVPR 2024 · similarity 0.87)
- VS: Reconstructing Clothed 3D Human from Single Image via Vertex Shift (CVPR 2024 · similarity 0.87)
- HiLo: Detailed and Robust 3D Clothed Human Reconstruction with High-and Low-Frequency Information of Parametric Models (CVPR 2024 · similarity 0.87)
- HISR: Hybrid Implicit Surface Representation for Photorealistic 3D Human Reconstruction (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.