Research map / Image generation / Text-to-image diffusion
Curriculum Direct Preference Optimization for Diffusion and Consistency Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Curriculum DPO, which orders preference pairs by reward-score gap (easy to hard) in batches, and applies it to Stable Diffusion v1.5 and Latent Consistency Model with LoRA, plus a Consistency-DPO adaptation.
Problem
Standard DPO for text-to-image diffusion and consistency models samples preference pairs randomly, ignoring their difficulty, which is suboptimal for alignment with text, aesthetics, and human preference.
Approach
Rank generated images per prompt with a reward model, then build pairs whose difficulty is set by their preference-score gap and feed them to DPO in batches from easy to hard (curriculum learning), and adapt DPO to consistency models (Consistency-DPO).
Key results
- On D1 with LCM, Curriculum DPO reaches HPSv2 0.3237 vs 0.2990 for DPO and 0.2912 for pre-trained LCM.
- On D2 (DrawBench) with SD, Curriculum DPO reaches text alignment 0.6234 vs 0.6075 for DPO and 0.5997 for baseline SD.
- Curriculum DPO with about 10x fewer training images matches DPO and DDPO performance, per the ablation on the number of images per prompt.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add curriculum learning to LoRA + DPO (ablation) | LCM · D1 | text alignment: 0.7502±0.0045 -> 0.7548±0.0041 | Helped |
| Add curriculum learning to LoRA + DPO (ablation) | LCM · D1 | aesthetics: 6.4741±0.0095 -> 6.6417±0.0083 | Helped |
| Add curriculum learning to LoRA + DPO (ablation) | LCM · D1 | human preference: 0.2990±0.0010 -> 0.3237±0.0012 | Helped |
| Add curriculum learning to LoRA + DPO (ablation) | SD · D1 | text alignment: 0.7614±0.0049 -> 0.7703±0.0036 | Helped |
| Add curriculum learning to LoRA + DPO (ablation) | SD · D1 | aesthetics: 5.7146±0.0162 -> 5.8232±0.0197 | Helped |
| Add curriculum learning to LoRA + DPO (ablation) | SD · D1 | human preference: 0.2827±0.0008 -> 0.2856±0.0015 | Helped |
| Add LoRA without DPO or curriculum learning (ablation) | LCM · D1 | text alignment: 0.7243±0.0048 -> 0.7151±0.0032 | Hurt |
| Add LoRA without DPO or curriculum learning (ablation) | LCM · D1 | aesthetics: 6.0490±0.0162 -> 5.7158±0.0154 | Hurt |
| Add LoRA without DPO or curriculum learning (ablation) | LCM · D1 | human preference: 0.2912±0.0021 -> 0.2881±0.0017 | Hurt |
| Add DPO to LoRA fine-tuning (ablation) | LCM · D1 | aesthetics: 5.7158±0.0154 -> 6.4741±0.0095 | Helped |
| Add LoRA without DPO or curriculum learning (ablation) | SD · D1 | text alignment: 0.6804±0.0052 -> 0.6852±0.0047 | No clear effect |
| Add curriculum learning to LoRA + DPO | LCM · D2 | aesthetics: 6.0430±0.0113 -> 6.1829±0.0128 | Helped |
Surprising findings
- Applying LoRA alone to LCM degrades performance on all three tasks.
Limitations
- Gains over DPO are modest in some settings (e.g., SD aesthetics on D1 is 5.8232 vs 5.8183 for DDPO), and results are averaged over only three runs.
- Evaluation is limited to three reward models, each evaluated on two datasets, with the reward model also used for training.
Experimental setup
Datasets: DrawBench Pick-a-Pic Black et al. SVO/animal prompt dataset (D1)
Benchmarks: DrawBench Pick-a-Pic subset (D3) Black et al. D1 prompts
Metrics: Sentence-BERT text alignment score LAION aesthetic score HPSv2 human preference score human rating 1-5
Baselines: DDPO DPO (Diffusion-DPO) pre-trained LCM pre-trained SD v1.5
Builds on: Diffusion-DPO DDPO Direct Preference Optimization Latent Consistency Model curriculum learning
Closest papers by meaning
- Boost Your Human Image Generation Model via Direct Preference Optimization (CVPR 2025 · similarity 0.90)
- DrPO: Drifting Preference Optimization for One-Step Generative Models (NeurIPS 2026 · similarity 0.89)
- D-Fusion: Direct Preference Optimization for Aligning Diffusion Models with Visually Consistent Samples (ICML 2025 · similarity 0.89)
- Diffusion Model Alignment Using Direct Preference Optimization (CVPR 2024 · similarity 0.89)
- Ranking-based Preference Optimization for Diffusion Models from Implicit User Feedback (NeurIPS 2025 · similarity 0.89)
- InPO: Inversion Preference Optimization with Reparametrized DDIM for Efficient Diffusion Model Alignment (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.