Research map / Image generation / Text-to-image diffusion
Boost Your Human Image Generation Model via Direct Preference Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HG-DPO, a three-stage curriculum DPO method that uses real human images as winning images (via easy, normal, hard stages) to improve realism of human image generation in latent diffusion models.
Problem
Diffusion-DPO and its variants use only generated images as winning samples, capping realism of human image generation at the level of generated images, and naively using real images as winners fails due to the large domain gap.
Approach
Use real images as DPO winning images, bridged by a three-stage curriculum: easy (AI-scored generated pairs with a statistics-matching loss against color shift), normal (SDEdit-style stochastic reconstruction intermediate domain), and hard (near-real intermediate winners), with no human feedback needed for dataset construction.
Key results
- HG-DPO reaches FID 29.41 and CLIP-IQA quality 0.9343 versus 35.35 and 0.8886 for Curriculum-DPO, and ranks best on CI-S (0.9858) and ATHEC (29.41).
- HG-DPO achieves HPS 0.2872 and ImageReward 0.7568, versus 0.2819 and -0.0665 for the base model.
- Naive single-stage DPO with real winners degrades results (FID 112.67, HPS 0.2425), showing the need for curriculum.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use HG-DPO instead of Curriculum-DPO | human image generation · Human image generation test set | FID: 35.35 -> 29.41 | Helped |
| use HG-DPO instead of AlignProp | human image generation · Human image generation test set | P-Score: 23.0202 -> 22.6043 | Hurt |
| train in one stage with real images as winners and generated images as losers (ablation) | Base (ϵbase) · Human image generation test set | FID: 37.34 -> 112.67 | Hurt |
| add the easy stage before normal and hard training (ablation) | HG-DPO · Human image generation test set | FID: 72.34 -> 29.41 | Helped |
| add the normal stage between easy and hard training (ablation) | HG-DPO · Human image generation test set | FID: 32.24 -> 29.41 | Helped |
| train sequentially through the three curriculum stages instead of end-to-end on their combined datasets (ablation) | HG-DPO · Human image generation test set | FID: 57.10 -> 29.41 | Helped |
| add text encoder training after the hard stage (ablation) | Hard (ϵH) · Human image generation test set | CLIP: 31.15 -> 31.57 | Helped |
| use an image pool with more than two candidates per prompt instead of exactly two (ablation) | easy stage · Human image generation test set | P-Score: 22.1939 -> 22.5688 | Helped |
| add statistics matching loss to the image-pool easy stage (ablation) | easy stage with N > 2 · Human image generation test set | Hue: 98.54 -> 27.94 | Helped |
| increase the weight of the original DPO regularization instead of using statistics matching loss (ablation) | easy stage with N > 2 · Human image generation test set | P-Score: 22.5688 -> 22.2506 | Hurt |
| adapt HG-DPO LoRA layers to InstantBooth | InstantBooth · Personalized text-to-image | FID: 39.61 -> 29.30 | Helped |
| apply SFT with hard-stage winning images after the normal stage instead of continuing with the hard stage (ablation) | Normal (ϵN) · Human image generation test set | FID: 28.66 -> 35.12 | Hurt |
Surprising findings
- Naively treating real images as winners with a single-stage DPO performs worse than the base model, with FID rising to 112.67.
- Supervised fine-tuning on hard-stage winners after earlier stages degrades results relative to the DPO-trained curriculum models.
Limitations
- Hard-stage training lacks a prompt-aware preference estimator, reducing image-text alignment, which needed a text encoder fix.
- Diversity is reduced in favor of image quality, as acknowledged in the analysis of the hard stage.
- Training uses an internal proprietary dataset that is not released.
Experimental setup
Datasets: internal human image dataset (~300k) HPD v2 Pick-a-Pic v2
Benchmarks: internal 5k human prompt test set personalized T2I evaluation with InstantBooth
Metrics: PickScore HPS-v2 ImageReward AestheticScore CLIP score FID CLIP-IQA quality CLIP-IQA sharpness ATHEC hue distance
Baselines: Diffusion-DPO NCP-DPO MAPO Curriculum-DPO AlignProp HPD v2-trained model Pick-a-Pic v2-trained model
Builds on: Diffusion-DPO Curriculum learning SDEdit TexForce InstantBooth
Closest papers by meaning
- Curriculum Direct Preference Optimization for Diffusion and Consistency Models (CVPR 2025 · similarity 0.90)
- DrPO: Drifting Preference Optimization for One-Step Generative Models (NeurIPS 2026 · similarity 0.86)
- CPO: Condition Preference Optimization for Controllable Image Generation (NeurIPS 2025 · similarity 0.85)
- Aesthetic Post-Training Diffusion Models from Generic Preferences with Step-by-step Preference Optimization (CVPR 2025 · similarity 0.85)
- Smoothed Preference Optimization via ReNoise Inversion for Aligning Diffusion Models with Varied Human Preferences (ICML 2025 · similarity 0.85)
- ImagenHub: Standardizing the evaluation of conditional image generation models (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.