Research map / Image generation / Fast diffusion sampling
When Preference Labels Fall Short: Aligning Diffusion Models from Real Data
Weiyan Chen, Weijian Deng, Yao Xiao, Weijie Tu, ZiYi Dong, Ibrahim Radwan, Liang Lin, Pengxu Wei
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use 512 real-image preference pairs instead of 851k annotated pairs (Diffusion-DPO) | SD-1.5 · Pick-a-Pic v2 test | HPSv3: 6.80 -> 7.33 | Improved |
| use 512 real-image preference pairs instead of Diffusion-DPO | SD-3.5-M · DrawBench | DeQA: 3.96 -> 4.26 | Improved |
| use real-data pairs vs Diffusion-DPO | SD-3.5-M · DrawBench | UnifiedReward: 3.47 -> 3.46 | No clear effect |
| add stage 1 distribution alignment before DPO (ablation) | two-stage method · Pick-a-Pic v2 test | HPSv3: 6.11 -> 7.33 | Improved |
| add stage 2 DPO on contrastive pairs after stage 1 (ablation) | two-stage method · Pick-a-Pic v2 test | PickScore: 20.87 -> 21.04 | Improved |
| increase training pairs from 256 to 512 (ablation) | two-stage method · Pick-a-Pic v2 test | PickScore, HPSv3, DeQA, aesthetic | Improved |
| increase training pairs beyond 512 (to 1024) (ablation) | two-stage method · Pick-a-Pic v2 test | PickScore, HPSv3, DeQA, aesthetic | No clear effect |
| apply real-data post-training on top of Diffusion-DPO | Diffusion-DPO (SD-1.5) · Pick-a-Pic v2 test | all metrics | Improved |
| apply real-data post-training on top of FlowGRPO | FlowGRPO (SD-3.5-M) · DrawBench | all metrics | Improved |
| use generated-image sources (Civitai-top) instead of real HPDv3 images (ablation) | two-stage method · Pick-a-Pic v2 test | HPSv3: 7.33 -> 8.67 | Improved |
| replace SD-1.5 inpainting with PixArt-alpha inpainting for negatives (ablation) | two-stage method · Pick-a-Pic v2 test | HPSv3: 7.33 -> 7.16 | No clear effect |
| replace inpainting with caption text-to-image negatives (ablation) | two-stage method · Pick-a-Pic v2 test | HPSv3: 7.33 -> 7.09 | No clear effect |
Closest papers by meaning
- Diffusion-NPO: Negative Preference Optimization for Better Preference Aligned Generation of Diffusion Models (ICLR 2025 · similarity 0.92)
- Calibrated Multi-Preference Optimization for Aligning Diffusion Models (CVPR 2025 · similarity 0.91)
- Smoothed Preference Optimization via ReNoise Inversion for Aligning Diffusion Models with Varied Human Preferences (ICML 2025 · similarity 0.90)
- Aligning Diffusion Models by Optimizing Human Utility (NeurIPS 2024 · similarity 0.90)
- Diffusion Model Alignment Using Direct Preference Optimization (CVPR 2024 · similarity 0.89)
- A Dense Reward View on Aligning Text-to-Image Diffusion with Preference (ICML 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
