Research map / Image generation / Fast diffusion sampling
Tuning Timestep-Distilled Diffusion Model Using Pairwise Sample Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes pairwise sample optimization (PSO), a DPO-style relative-likelihood objective that directly fine-tunes timestep-distilled diffusion models (SDXL-DMD2, SDXL-Turbo, SDXL-LCM) for human preference, style transfer, and concept customization without re-distillation.
Problem
Fine-tuning timestep-distilled few-step diffusion models with the standard diffusion loss degrades them into blurry outputs, and re-running distillation after each customization is computationally expensive.
Approach
Treat the distilled sampler as an MDP and maximize the likelihood margin between target (data) trajectories and reference trajectories sampled from the current model, regularized to the pre-trained distilled model; offline and online variants unify DiffusionDPO and D3PO-style objectives as special cases.
Key results
- Online-PSO on SDXL-DMD2 reaches PickScore 22.73 on Pick-a-Pic test and 23.29 on PartiPrompts at 4 steps, above SDXL-DPO (22.60 / 22.92) and SDXL-DMD2 (22.35 / 22.99).
- Offline-PSO and Online-PSO improve SDXL-Turbo in both 1-step and 4-step generation over the base distilled model on PickScore, ImageReward, and Aesthetic Score.
- On DreamBooth concept customization with SDXL-Turbo, PSO gets CLIP-T 0.271 (vs 0.267 for Dreambooth w/ SDXL-Turbo) and DINO 0.593 (vs 0.151), with the multi-step Dreambooth w/ SDXL at DINO 0.610.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fine-tune SDXL-DMD2 with Offline-PSO using offline preference pairs | SDXL-DMD2 · Pick-a-Pic test | PickScore: 22.35 -> 22.46 | Helped |
| Fine-tune SDXL-DMD2 with Online-PSO using reward-ranked samples | SDXL-DMD2 · Pick-a-Pic test | PickScore: 22.35 -> 22.73 | Helped |
| Fine-tune SDXL-DMD2 with Offline-PSO using offline preference pairs | SDXL-DMD2 · Parti-Prompts | CLIP Score: 0.3607 -> 0.3649 | Helped |
| Fine-tune SDXL-DMD2 with Online-PSO using reward-ranked samples | SDXL-DMD2 · Parti-Prompts | ImageReward: 1.0713 -> 1.1702 | Helped |
| Fine-tune SDXL-Turbo with Online-PSO | SDXL-Turbo-4step · Pick-a-Pic test, 4-step generation | PickScore: 22.22 -> 22.71 | Helped |
| Fine-tune SDXL-Turbo with Online-PSO | SDXL-Turbo-1step · Pick-a-Pic test, 1-step generation | ImageReward: 0.8830 -> 0.9113 | Helped |
| Fine-tune SDXL-Turbo with Online-PSO | SDXL-Turbo-4step · Parti-Prompts, 4-step generation | PickScore: 22.88 -> 23.23 | Helped |
| Fine-tune SDXL-Turbo with Online-PSO | SDXL-Turbo-1step · Parti-Prompts, 1-step generation | ImageReward: 0.9246 -> 0.9762 | Helped |
| Apply DMD2 LoRA to preference-tuned SDXL-DPO rather than fine-tuning the distilled model | SDXL-DPO + DMD2-LoRA · Pick-a-Pic test | PickScore: 22.35 -> 22.20 | Hurt |
| Use Dreambooth customization on SDXL-Turbo instead of PSO | SDXL-Turbo · Concept customization | DINO: 0.593 -> 0.151 | Hurt |
| Use PSO for SDXL-Turbo concept customization instead of vanilla Dreambooth tuning | SDXL-Turbo · Concept customization | DINO: 0.151 -> 0.593 | Helped |
| Remove the reference model and add prior-preservation loss (ablation) | Offline-PSO for SDXL-Turbo concept customization · Concept customization | performance | Helped |
Surprising findings
- Naively fine-tuning timestep-distilled models with the standard diffusion loss yields blurry, degraded outputs.
- Online-PSO on the distilled SDXL-DMD2 surpasses the multi-step SDXL-DPO on PickScore, despite using only 4 steps.
Limitations
- Authors note style transfer and concept customization results are preliminary and image quality is not yet competitive; further work on loss design and regularization is needed.
- Evaluation relies on automatic metrics (PickScore, CLIP, ImageReward, aesthetic score) without human evaluation.
Experimental setup
Datasets: Pick-a-Pic v2 Pokemon BLIP captions dataset DreamBooth
Benchmarks: Pick-a-Pic test PartiPrompts DreamBooth concept customization
Metrics: PickScore CLIP Score ImageReward Aesthetic Score CLIP-T DINO CLIP-I
Baselines: SDXL SDXL-DPO SDXL-SFT SDXL-RPO SDXL-SPO SDXL-MaPO SDXL-DMD2 SDXL-DPO + DMD2-LoRA Dreambooth w/ SDXL Dreambooth w/ SDXL-Turbo
Builds on: Diffusion-DPO D3PO DMD2 SDXL-Turbo LCM DreamBooth
Closest papers by meaning
- Diffusion Model is Effectively Its Own Teacher (CVPR 2025 · similarity 0.90)
- Alleviating Exposure Bias in Diffusion Models through Sampling with Shifted Time Steps (ICLR 2024 · similarity 0.90)
- Multistep Distillation of Diffusion Models via Moment Matching (NeurIPS 2024 · similarity 0.90)
- Simple and Fast Distillation of Diffusion Models (NeurIPS 2024 · similarity 0.89)
- Faster Diffusion: Rethinking the Role of the Encoder for Diffusion Model Inference (NeurIPS 2024 · similarity 0.89)
- Adaptive Non-Uniform Timestep Sampling for Accelerating Diffusion Model Training (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.