Research map / Image generation / Fast diffusion sampling
Offline Preference Optimization for Rectified Flow with Noise-Tracked Pairs
Yunhong Lu, Qichao Wang, Hengyuan Cao, Xiaoyin Xu, Min Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add stored prior noise and noise-image interpolation to DPO (ablation) | Diffusion-DPO on FLUX.1-dev · HPDv2 | HPSv2.1: 30.84 -> 31.08 | Improved |
| add dynamic regularization on top of prior noise (ablation) | PNAPO on FLUX.1-dev · HPDv2 | HPSv2.1: 31.08 -> 31.71 | Improved |
| add dynamic regularization without prior noise (ablation) | Diffusion-DPO on FLUX.1-dev · HPDv2 | HPSv2.1: 30.84 -> 30.96 | Improved |
| add reward-gap controller f(delta r) to beta (ablation) | PNAPO · HPDv2 | HPSv2.1: 31.08 -> 31.66 | Improved |
| add training-progress annealing g(n) to beta (ablation) | PNAPO · HPDv2 | HPSv2.1: 31.08 -> 31.13 | Improved |
| use HPSv2.1 as reward model instead of ImageReward (ablation) | PNAPO · HPDv2 | HPSv2.1: 31.10 -> 31.71 | Improved |
| use Aesthetic reward model instead of HPSv2.1 (ablation) | PNAPO · HPDv2 | Aesthetic: 6.475 -> 6.509 (Aesthetic), CLIP 34.71 -> 34.57 | Mixed |
| lower first threshold n1 from 1000 to 500 (ablation) | dynamic regularization schedule · HPDv2 | HPSv2.1: 31.71 -> 30.96 | Hurt |
| increase n2 from 2000 to 4000 (ablation) | dynamic regularization schedule · HPDv2 | HPSv2.1: 31.71 -> 31.62 | No clear effect |
| train with PNAPO instead of Diffusion-DPO | FLUX.1-dev · training cost | GPU-hours: ~422.4 -> ~35.2 | Improved |
| apply PNAPO to SD3-M | SD3-M · GenEval | overall: 0.68 -> 0.73 | Improved |
| apply PNAPO to FLUX on counting attribute | FLUX.1-dev · GenEval | overall: 0.65 -> 0.69 | Improved |
Closest papers by meaning
- $\alpha$-DPO: Robust Preference Alignment for Diffusion Models via $\alpha$ Divergence (ICLR 2026 · similarity 0.91)
- Diffusion-SDPO: Safeguarded Direct Preference Optimization for Diffusion Models (ECCV 2026 · similarity 0.90)
- Smoothed Preference Optimization via ReNoise Inversion for Aligning Diffusion Models with Varied Human Preferences (ICML 2025 · similarity 0.89)
- Flow-GRPO: Training Flow Matching Models via Online RL (NeurIPS 2025 · similarity 0.88)
- Calibrated Multi-Preference Optimization for Aligning Diffusion Models (CVPR 2025 · similarity 0.87)
- Ranking-based Preference Optimization for Diffusion Models from Implicit User Feedback (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
