Research map / Image generation / Text-to-image diffusion
Rethinking Direct Preference Optimization in Diffusion Models
Junyong Kang, Seohyun Lim, Kyungjune Baek, Hyunjung Shim
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add timestep-aware optimization on top of regularized reference-update exploration (ablation) | Diffusion-DPO with reference update · Pick-a-Pic v2 test (SD1.5) | PickScore: 21.88 -> 21.93 | Improved |
| add timestep-aware optimization alone to Diffusion-DPO (ablation) | Diffusion-DPO · Pick-a-Pic v2 test (SD1.5) | PickScore: 21.36 -> 21.10 | Hurt |
| add regularized reference-model update (exploration) to Diffusion-DPO (ablation) | Diffusion-DPO · Pick-a-Pic v2 test (SD1.5) | PickScore: 21.36 -> 21.88 | Improved |
| increase learning rate 1e-8 to 5e-8 to encourage exploration (ablation) | Diffusion-DPO · Pick-a-Pic v2 test (SD1.5) | PickScore: 21.36 -> 19.75 | Hurt |
| reduce beta from 5000 to 1000 (ablation) | Diffusion-DPO · Pick-a-Pic v2 test (SD1.5) | PickScore: 21.36 -> 21.26 | No clear effect |
| re-initialize reference to pretrained model instead of freezing at divergence boundary (ablation) | reference update exploration · Pick-a-Pic v2 test (SD1.5) | PickScore: 21.50 (Re-Init) vs 21.93 (Ours) | Mixed |
| add reward scale schedule lambda(t) to timestep oversampling (gamma 0.9) (ablation) | timestep-aware strategy · Pick-a-Pic v2 test (SD1.5) | PickScore: 21.82 -> 21.93 | Improved |
| oversample early timesteps, gamma 0.9 vs 0.8 (ablation) | timestep-aware strategy · Pick-a-Pic v2 test (SD1.5) | ImageReward: 0.52 -> 0.64 | Improved |
| add divergence regularization to reference update across update periods tau 16/32/64 (ablation) | reference update · Pick-a-Pic v2, PartiPrompts, HPDv2 (SD1.5) | PickScore | Improved |
| full method vs strong baselines on SDXL, Aesthetic metric vs MaPO | Ours · Pick-a-Pic v2 test (SDXL) | Aesthetic win rate: 45.12 | Hurt |
Closest papers by meaning
- Diffusion Model Alignment Using Direct Preference Optimization (CVPR 2024 · similarity 0.95)
- InPO: Inversion Preference Optimization with Reparametrized DDIM for Efficient Diffusion Model Alignment (CVPR 2025 · similarity 0.94)
- Smoothed Preference Optimization via ReNoise Inversion for Aligning Diffusion Models with Varied Human Preferences (ICML 2025 · similarity 0.93)
- Calibrated Multi-Preference Optimization for Aligning Diffusion Models (CVPR 2025 · similarity 0.91)
- Personalized Preference Fine-tuning of Diffusion Models (CVPR 2025 · similarity 0.91)
- Ranking-based Preference Optimization for Diffusion Models from Implicit User Feedback (NeurIPS 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
