Research map / Image generation / Text-to-image diffusion
CRAFT: Aligning Diffusion Models with Fine-Tuning Is Easier Than You Think
Zening Sun, Zhengpeng Xie, Lichen Bai, Shitong Shao, Shuo Yang, Zeke Xie
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| fine-tune on top-100 reward-selected samples instead of all 2,312 filtered samples (ablation) | CRAFT (SDXL) · Pick-a-Pic (SDXL) | HPS: 32.20 -> 33.23 | Improved |
| select top-100 instead of 500 random samples (ablation) | CRAFT (SDXL) · Pick-a-Pic (SDXL) | HPS: 33.10 -> 33.23 | Improved |
| select highest-scoring samples instead of lowest-scoring 500 (ablation) | CRAFT (SDXL) · Pick-a-Pic (SDXL) | HPS: 31.86 -> 33.23 | Improved |
| add PickScore to the composite reward (HPS+AES -> HPS+Pick+AES) (ablation) | CRAFT reward filtering · Pick-a-Pic (SDXL) | HPS: 30.93 -> 32.20 | Improved |
| use full composite reward instead of PickScore alone (ablation) | CRAFT reward filtering · Pick-a-Pic (SDXL) | HPS: 31.71 -> 32.20 | Improved |
| use composite reward instead of HPS alone (ablation) | CRAFT reward filtering · Pick-a-Pic (SDXL) | HPS: 30.25 -> 32.20 | Improved |
| CRAFT with 100 samples instead of SPO (4,000 samples) on SDXL | SDXL alignment · HPDv2 (SDXL) | HPSv2.1: 32.32 -> 32.67 | Improved |
| CRAFT instead of SPO on SD1.5 aesthetic score | SD1.5 alignment · HPDv2 (SD1.5) | AES: 5.758 -> 5.699 | Hurt |
| CRAFT instead of SPO on compositional GenEval | SD1.5 alignment · GenEval (SD1.5) | overall: 44.04 -> 45.82 | Improved |
| CRAFT instead of SPO for training compute | SDXL alignment · Fine-tuning cost (SDXL) | GPU hours: 63.0 -> 4.0 | Improved |
Closest papers by meaning
- Directly Fine-Tuning Diffusion Models on Differentiable Rewards (ICLR 2024 · similarity 0.92)
- Using Human Feedback to Fine-tune Diffusion Models without Any Reward Model (CVPR 2024 · similarity 0.92)
- Calibrated Multi-Preference Optimization for Aligning Diffusion Models (CVPR 2025 · similarity 0.91)
- Self-Play Fine-tuning of Diffusion Models for Text-to-image Generation (NeurIPS 2024 · similarity 0.90)
- InPO: Inversion Preference Optimization with Reparametrized DDIM for Efficient Diffusion Model Alignment (CVPR 2025 · similarity 0.90)
- Reward Score Matching: Unifying Reward-based Fine-tuning for Flow and Diffusion Models (NeurIPS 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
