Research map / Image generation / Editing and style transfer
Disentangled Pose and Appearance Guidance for Multi-Pose Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a multi-pose human image generation framework that disentangles a transformer-based global-aware pose generation module from a diffusion-based appearance refinement stage with an Appearance Adapter, evaluated on UBC Fashion and TikTok.
Problem
Human pose generation (transforming a reference person image to multiple target poses) suffers from overfitting and poor control of non-rigid deformations and appearance consistency, since existing diffusion methods entangle spatial pose transformation with texture generation.
Approach
Separates pose control from appearance generation: a Global-aware Transformer Decoder iteratively produces pose embeddings for all target poses with a similarity-based global-aware block, and a latent diffusion model conditioned on these embeddings is refined by a multi-scale Appearance Adapter.
Key results
- On UBC Fashion, the method reaches SSIM 0.941, PSNR 25.655, FID 11.487 and LPIPS 0.0497, outperforming CFLD (FID 12.810) and MagicPose.
- On TikTok, FID 103.708 vs. 105.631 for CFLD, the best prior FID, while CFLD keeps a higher SSIM (0.729 vs 0.753 for ours, so ours is 0.753 and higher on SSIM).
- User study with 30 volunteers: 61.4% preference for the proposed method versus 16.7% for CFLD.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add Global-aware Pose Generation (ablation) | Ours · UBC Fashion | SSIM: 0.9362 -> 0.9411 | Helped |
| Use the Multi-stage Image Encoder instead of its replacement (ablation) | Ours · UBC Fashion | SSIM: 0.9344 -> 0.9411 | Helped |
| Add Appearance Adapter (ablation) | Ours · UBC Fashion | SSIM: 0.9384 -> 0.9411 | Helped |
| Fine-tune the VAE (ablation) | Ours · UBC Fashion | SSIM: 0.9386 -> 0.9411 | Helped |
| Add reconstruction loss Lrec (ablation) | Ours · UBC Fashion | SSIM: 0.9403 -> 0.9411 | Helped |
| Use the proposed decoder instead of alternative architecture (a) (ablation) | Global-aware Transformer Decoder · UBC Fashion | SSIM: 0.9389 -> 0.9411 | Helped |
| Use the proposed decoder instead of alternative architecture (b) (ablation) | Global-aware Transformer Decoder · UBC Fashion | SSIM: 0.9381 -> 0.9411 | Helped |
| Use Ours rather than CFLD | Human pose generation · UBC Fashion | SSIM: 0.889 -> 0.941 | Helped |
| Use Ours rather than CFLD | Human pose generation · TikTok | SSIM: 0.729 -> 0.753 | Helped |
| Increase the number of generated images beyond the training setting | Ours · UBC Fashion | LPIPS | Hurt |
Surprising findings
- Replacing the multi-stage image encoder with a CLIP image encoder improved FID slightly while lowering SSIM and PSNR, indicating metric trade-offs.
Limitations
- Performance declines slightly as the number of generated images increases beyond the training setting.
- On TikTok, SSIM and LPIPS are not uniformly best compared with CFLD and MagicPose.
Experimental setup
Datasets: UBC Fashion TikTok
Benchmarks: UBC Fashion TikTok
Metrics: SSIM PSNR LPIPS FID L1 user preference
Baselines: PISE DPTN NTED BiGraphGAN PIDM CFLD MagicPose
Builds on: Stable Diffusion DPTN PIDM MagicPose Swin Transformer
Closest papers by meaning
- UniPose: A Unified Multimodal Framework for Human Pose Comprehension, Generation and Editing (CVPR 2025 · similarity 0.87)
- ChatPose: Chatting about 3D Human Pose (CVPR 2024 · similarity 0.86)
- DAGSM: Disentangled Avatar Generation with GS-enhanced Mesh (CVPR 2025 · similarity 0.86)
- IMAGPose: A Unified Conditional Framework for Pose-Guided Person Generation (NeurIPS 2024 · similarity 0.86)
- Direct May Not Be the Best: An Incremental Evolution View of Pose Generation (AAAI 2024 · similarity 0.86)
- Semantic-Aware Human Object Interaction Image Generation (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.