Research map / Image generation / Editing and style transfer
Advancing Pose-Guided Image Synthesis with Progressive Conditional Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Progressive Conditional Diffusion Models (PCDMs), a three-stage diffusion pipeline (prior, inpainting, refining) for pose-guided person image synthesis, evaluated on DeepFashion and Market-1501.
Problem
Pose-guided person image synthesis must generate a person in a target pose from a source image and target pose alone, which is hard because source and target poses are misaligned, leading to distorted textures and weak realism.
Approach
Decompose generation into three diffusion stages: a prior model predicts the CLIP global embedding of the target image from pose coordinates and source image; an inpainting diffusion model builds dense source-target correspondence conditioned on that embedding; and a refining diffusion model takes the coarse image as a condition to restore texture.
Key results
- On DeepFashion (256x176), PCDMs reach SSIM 0.7444, LPIPS 0.1365, FID 7.4734, beating PIDM on SSIM and LPIPS but not FID (PIDM 6.3671).
- On Market-1501 (128x64), PCDMs achieve best SSIM 0.3169, LPIPS 0.2238, FID 13.897, outperforming PIDM (14.451 FID).
- User study on DeepFashion: 56.2% G2R and 44.1% Jab score for PCDMs.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the refining conditional diffusion model (ablation) | PCDMs · DeepFashion 256×176 | SSIM: 0.7357 -> 0.7444 | Helped |
| add the refining conditional diffusion model (ablation) | PCDMs · DeepFashion 256×176 | LPIPS: 0.1426 -> 0.1365 | Helped |
| add the refining conditional diffusion model (ablation) | PCDMs · DeepFashion 256×176 | FID: 7.7815 -> 7.4734 | Hurt |
| add the refining conditional diffusion model (ablation) | PCDMs · DeepFashion 512×352 | SSIM: 0.7532 -> 0.7601 | Helped |
| add the refining conditional diffusion model (ablation) | PCDMs · DeepFashion 512×352 | LPIPS: 0.1583 -> 0.1475 | Helped |
| add the refining conditional diffusion model (ablation) | PCDMs · DeepFashion 512×352 | FID: 7.8422 -> 7.5519 | Hurt |
| add the refining conditional diffusion model (ablation) | PCDMs · Market-1501 128×64 | SSIM: 0.3107 -> 0.3169 | Helped |
| add the refining conditional diffusion model (ablation) | PCDMs · Market-1501 128×64 | LPIPS: 0.2329 -> 0.2238 | Helped |
| add the refining conditional diffusion model (ablation) | PCDMs · Market-1501 128×64 | FID: 14.162 -> 13.897 | Helped |
| add the prior conditional diffusion model to inpainting (ablation) | PCDMs B1 · DeepFashion | pose structure alignment | Helped |
| add the refining conditional diffusion model to inpainting (ablation) | PCDMs B1 · DeepFashion | detail texture quality | Helped |
| add the prior, inpainting, and refining stages together (ablation) | PCDMs · DeepFashion | visual quality | Helped |
Surprising findings
- PCDMs achieve a higher FID than PIDM on DeepFashion at 256x176 and 512x352 resolutions, which the authors acknowledge.
Limitations
- Two extra prior and refining stages increase computational cost and inference time.
- FID is worse than PIDM on DeepFashion.
Experimental setup
Datasets: DeepFashion Market-1501
Benchmarks: DeepFashion Market-1501
Metrics: SSIM LPIPS FID R2G G2R Jab Rank-1
Baselines: Def-GAN PATN ADGAN PISE GFLA DPTN CASD NTED PIDM CocosNet2 PTN
Builds on: unCLIP latent diffusion models ControlNet classifier-free guidance PIDM
Closest papers by meaning
- Coarse-to-Fine Latent Diffusion for Pose-Guided Person Image Synthesis (CVPR 2024 · similarity 0.91)
- Multi-focal Conditioned Latent Diffusion for Person Image Synthesis (CVPR 2025 · similarity 0.90)
- Cross-view Masked Diffusion Transformers for Person Image Synthesis (ICML 2024 · similarity 0.89)
- IMAGPose: A Unified Conditional Framework for Pose-Guided Person Generation (NeurIPS 2024 · similarity 0.89)
- DiffPortrait3D: Controllable Diffusion for Zero-Shot Portrait View Synthesis (CVPR 2024 · similarity 0.87)
- DiffusionRegPose: Enhancing Multi-Person Pose Estimation using a Diffusion-Based End-to-End Regression Approach (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.