Research map / 3D vision / 3D generation
Pippo: High-Resolution Multi-View Humans from a Single Image
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Pippo is a multi-view diffusion transformer that generates 1K-resolution, 3D-consistent turnaround views of a person from a single unposed photo, trained in three stages on 3B in-the-wild images and proprietary studio captures.
Problem
Generating dense, high-resolution, multi-view consistent images of a human from a single casually taken photo, without body priors or camera parameters of the input, while scaling beyond scarce studio multi-view data.
Approach
A DiT-based multi-view diffusion model pre-trained on unlabeled in-the-wild human images, then mid-trained on many low-res views with coarse camera MLP encoding and post-trained at high resolution with ControlMLP pixel-aligned controls (SIREN-encoded Plücker rays and a Spatial Anchor), plus a RE@SG reprojection-error metric for 3D consistency.
Key results
- Pippo (P3@1K) achieves RE@SG 3.0 on iPhone head-only faces and 1.7 on iPhone full-body, versus 4.1-5.3 for Era3D, Wonder3D, and MV-Adapter on the iPhone full-body benchmark.
- On the iPhone full-body benchmark, Pippo reaches face similarity 58.0 and body similarity 68.1, higher than all four multi-view baselines (best baseline face 43.0).
- Ablations show removing mid-training raises RE@SG from 3.3 to 5.6-ish range at 128 resolution and removing the spatial anchor raises RE@SG to 11.5 (P3@128).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use human-centric filtering for pretraining (ablation) | Pippo pretraining · iPhone dataset | FID: 86.838 -> 75.639 | Helped |
| remove human-centric filtering (ablation) | Pippo pretraining · iPhone dataset | FID: 75.639 -> 86.838 | Hurt |
| use image-conditioned instead of text-conditioned pretraining (ablation) | Pippo pretraining · iPhone dataset | FID: 109.720 -> 75.639 | Helped |
| remove mid-training (ablation) | Pippo · Head-only dataset | RE@SG: 2.0? no | Hurt |
| remove Spatial Anchor (ablation) | Pippo · Head-only dataset | RE@SG: 2.7 -> 11.5 | Hurt |
| remove Plücker coordinates (ablation) | Pippo · Head-only dataset | RE@SG: 3.3 -> 4.2 | Hurt |
| remove pretraining (ablation) | Pippo · Head-only dataset | RE@SG: 3.4 -> 5.9 | Hurt |
| replace self-attention conditioning with cross-attention (ablation) | Pippo · Head-only dataset | RE@SG: 3.4 -> 3.5 | Hurt |
| randomize the input reference viewpoint (ablation) | Pippo · Head-only dataset | RE@SG: 3.4 -> 5.8 | Hurt |
| increase pretraining data from 30M to 2.1B images (ablation) | Pippo · Full-body dataset | Face similarity: 30.6 -> 37.6 | Helped |
| increase pretraining data from 2.1B to Humans-3B (100%) (ablation) | Pippo · Full-body dataset | Face similarity: 37.6 -> 41.7 | Helped |
| encode camera with MLP and overfit (ablation) | Mid-trained multi-view model · Head-only scene validation views | PSNR: 19.70 -> 17.95 | Hurt |
Surprising findings
- Generating many views at low resolution in mid-training then fine-tuning at high resolution was needed; skipping mid-training degraded multi-view consistency substantially (RE@SG 5.6 vs 3.3).
Limitations
- Number of simultaneously generated views is limited by context length and GPU memory (post-training uses only 2 views at 1K).
- Training relies on large proprietary studio and in-the-wild datasets that are not released.
- Currently generates images only, not multi-view consistent video.
Experimental setup
Datasets: Humans-3B Proprietary full-body studio captures Proprietary head-only studio captures iPhone casual dataset Ava-256 Goliath
Benchmarks: iPhone dataset Studio held-out splits Ava-256 Goliath
Metrics: RE@SG PSNR SSIM LPIPS FID Face similarity (FaceNet) Body similarity (CLIP)
Baselines: SiTH DiffPortrait3D MV-Adapter Era3D Wonder3D
Builds on: DiT DALL-E 2 image decoder ControlNet Stable Diffusion 3 DDPM
Closest papers by meaning
- MEAT: Multiview Diffusion Model for Human Generation on Megapixels with Mesh Attention (CVPR 2025 · similarity 0.85)
- NAMVIS: Next-Scale Autoregressive Multi-View Image Synthesis (NeurIPS 2026 · similarity 0.85)
- MagicMan: Generative Novel View Synthesis of Humans with 3D-Aware Diffusion and Iterative Refinement (AAAI 2025 · similarity 0.84)
- MultiPly: Reconstruction of Multiple People from Monocular Video in the Wild (CVPR 2024 · similarity 0.84)
- UniMC: Taming Diffusion Transformer for Unified Keypoint-Guided Multi-Class Image Generation (ICML 2025 · similarity 0.83)
- MVSplat360: Feed-Forward 360 Scene Synthesis from Sparse Views (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.