Research map / 3D vision / Face and shape
Total Selfie: Generating Full-Body Selfies
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Generates full-body selfies from four close-up selfies, a background image, and a target pose by training a selfie-conditioned Paint-By-Example-style inpainting diffusion model on synthetic data and fine-tuning per capture.
Problem
Self-captured selfies show only the upper body with close-up perspective distortion; the task is to synthesize the full-body photo someone else would take of the user in a desired scene and target pose.
Approach
Builds a synthetic paired dataset by simulating selfie views from third-person full-body images via homography warping, trains an image-conditioned inpainting diffusion model (Paint-By-Example adapted to four selfie embeddings), then fine-tunes per capture with a ControlNet-based pose guidance and face/shoe appearance refinement.
Key results
- Total Selfie achieves LPIPS 0.187, SSIM 0.708, PSNR 17.01, FID 139.4, the best among compared methods (best baseline DreamBooth: LPIPS 0.224, FID 159.2).
- Ablations show per-capture fine-tuning and appearance refinement improve results; full pipeline FID 139.4 vs 154.1 for variant without fine-tuning and refinement.
- Paint-By-Example, DisCo, and LaDI-VTON perform worse (FID 184.6, 245.7, 172.8).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add per-capture fine-tuning and appearance refinement (ablation) | Total Selfie · 17 captures | LPIPS: 0.218 -> 0.187 | Helped |
| add appearance refinement (ablation) | Total Selfie with per-capture fine-tuning · 17 captures | LPIPS: 0.190 -> 0.187 | Helped |
| add face undistortion (ablation) | Total Selfie · 17 captures | LPIPS: 0.188 -> 0.187 | Helped |
| use Total Selfie instead of DreamBooth | full-body selfie generation · 17 captures | LPIPS: 0.224 -> 0.187 | Helped |
| use Total Selfie instead of LaDI-VTON | full-body selfie generation · 17 captures | LPIPS: 0.263 -> 0.187 | Helped |
| use Total Selfie instead of Paint-By-Example | full-body selfie generation · 17 captures | LPIPS: 0.252 -> 0.187 | Helped |
| use Total Selfie instead of DisCo | full-body selfie generation · 17 captures | LPIPS: 0.287 -> 0.187 | Helped |
Limitations
- Cannot accurately generate hard shadows under strong sunlight.
- Body shape and scale are inferred only from the background and selfies, sometimes inaccurate.
- Identity details of small faces are lost due to VAE limitations, requiring appearance refinement.
Experimental setup
Datasets: Synthetic selfie-to-full-body pairs generated with Stable Diffusion and OpenPose ControlNet FFHQ 3D GAN rendered heads for face undistortion
Benchmarks: Custom evaluation set of 12 subjects, 17 captures
Metrics: LPIPS SSIM PSNR FID
Baselines: Paint-By-Example DisCo LaDI-VTON DreamBooth
Builds on: Paint-By-Example ControlNet DreamBooth Latent Diffusion Stable Diffusion
Closest papers by meaning
- Egocentric Whole-Body Motion Capture with FisheyeViT and Diffusion-Based Motion Refinement (CVPR 2024 · similarity 0.80)
- Visual Persona: Foundation Model for Full-Body Human Customization (CVPR 2025 · similarity 0.80)
- MagicPose: Realistic Human Poses and Facial Expressions Retargeting with Identity-aware Diffusion (ICML 2024 · similarity 0.80)
- IDOL: Instant Photorealistic 3D Human Creation from a Single Image (CVPR 2025 · similarity 0.79)
- IMAGPose: A Unified Conditional Framework for Pose-Guided Person Generation (NeurIPS 2024 · similarity 0.79)
- PSHuman: Photorealistic Single-image 3D Human Reconstruction using Cross-Scale Multiview Diffusion and Explicit Remeshing (CVPR 2025 · similarity 0.79)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.