Research map / Image generation / Editing and style transfer
Controllable Human Image Generation with Personalized Multi-Garments
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BootComp, which bootstraps a large synthetic dataset of humans paired with multiple extracted garment images via a learned garment decomposition model plus similarity filtering, then trains an SDXL-based extended-self-attention composition module for multi-garment human image generation.
Problem
Controllable human image generation conditioned on multiple reference garments is bottlenecked by the lack of training data: paired datasets of a human with every garment they wear are hard to collect, and segmentation-based curation causes copy-and-paste and subject blending artifacts.
Approach
Train a decomposition network that maps a segmented garment in a person image to a product-view garment image, use it to extract all garments from large numbers of human images to build a synthetic multi-garment paired dataset, filter by DreamSim similarity, and train a composition module (trainable encoder + frozen SDXL generator with extended self-attention) on it.
Key results
- BootComp achieves MP-LPIPS 0.187 versus 0.267 for Parts2Whole and 0.276 for MIP-Adapter, a roughly 30% improvement.
- BootComp obtains FID 27.63 versus 28.39 (Parts2Whole) and 59.99 (MIP-Adapter), and DINO 0.379 versus 0.362.
- Training on synthetic pairs instead of segmented pairs improves MP-LPIPS from 0.374 to 0.197 and FID from 59.27 to 29.41 (SD v1.5 ablation).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use BootComp instead of Parts2Whole | Multi-garment human image generation · Multi-garment human image generation | MP-LPIPS: 0.267 -> 0.187 | Helped |
| Use BootComp instead of Parts2Whole | Multi-garment human image generation · Multi-garment human image generation | DINO: 0.362 -> 0.379 | Helped |
| Use BootComp instead of Parts2Whole | Multi-garment human image generation · Multi-garment human image generation | M-DINO: 0.036 -> 0.046 | Helped |
| Use BootComp instead of Parts2Whole | Multi-garment human image generation · Multi-garment human image generation | FID: 28.39 -> 27.63 | Helped |
| Train on synthetic pairs instead of segmented pairs (ablation) | Composition module · Multi-garment human image generation | MP-LPIPS: 0.374 -> 0.197 | Helped |
| Train on synthetic pairs instead of segmented pairs (ablation) | Composition module · Multi-garment human image generation | DINO: 0.284 -> 0.365 | Helped |
| Train on synthetic pairs instead of segmented pairs (ablation) | Composition module · Multi-garment human image generation | M-DINO: 0.025 -> 0.043 | Helped |
| Train on synthetic pairs instead of segmented pairs (ablation) | Composition module · Multi-garment human image generation | FID: 59.27 -> 29.41 | Helped |
| Increase dataset size from 5K to 50K (ablation) | Composition module · Multi-garment human image generation | DINO: 0.337 -> 0.360 | Helped |
| Increase dataset size from 5K to 50K (ablation) | Composition module · Multi-garment human image generation | M-DINO: 0.248 -> 0.285 | Helped |
| Increase dataset size from 5K to 50K (ablation) | Composition module · Multi-garment human image generation | FID: 34.15 -> 25.88 | Helped |
| Apply filtering at τ=0.4 instead of no filtering (ablation) | Synthetic data generation · Multi-garment human image generation | DINO: 0.338 -> 0.360 | Helped |
Limitations
- Filtering with threshold 0.4 keeps only 54K of 240K candidate pairs, discarding most data.
- Evaluation relies on a manually curated dataset with no standard benchmark for controllable human generation.
Experimental setup
Datasets: VITON-HD DressCode LAION-Fashion DeepFashion
Benchmarks: DressCode test set (garment sets) VITON-HD test set DeepFashion test set
Metrics: FID MP-LPIPS DINO M-DINO
Baselines: MIP-Adapter Parts2Whole
Builds on: Stable Diffusion XL IP-Adapter DreamSim Parts2Whole
Closest papers by meaning
- DreamFit: Garment-Centric Human Generation via a Lightweight Anything-Dressing Encoder (AAAI 2025 · similarity 0.89)
- FashionChameleon: Towards Real-Time and Interactive Human-Garment Video Customization (NeurIPS 2026 · similarity 0.88)
- FashionR2R: Texture-preserving Rendered-to-Real Image Translation with Diffusion Models (NeurIPS 2024 · similarity 0.88)
- AnyDressing: Customizable Multi-Garment Virtual Dressing via Latent Diffusion Models (CVPR 2025 · similarity 0.88)
- IMAGDressing-v1: Customizable Virtual Dressing (AAAI 2025 · similarity 0.87)
- FashionTailor: Controllable Clothing Editing for Human Images with Appearance Preserving (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.