Research map / Reinforcement learning / RL from feedback
Reinforcing Diffusion Models by Direct Group Preference Optimization
Yihong Luo, Tianyang Hu, Jing Tang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace Flow-GRPO with DGPO | SD3.5-M · GenEval | overall score: 0.95 -> 0.97 | Improved |
| replace Flow-GRPO with DGPO | SD3.5-M · visual text rendering | OCR accuracy: 0.92 -> 0.96 | Improved |
| replace Flow-GRPO with DGPO | SD3.5-M · human preference (PickScore) | PickScore: 23.31 -> 23.89 | Improved |
| replace Flow-GRPO with DGPO | SD3.5-M · DrawBench out-of-domain (GenEval-trained) | Aesthetic score: 5.25 -> 5.31 | No clear effect |
| train with DGPO rather than Flow-GRPO | SD3.5-M · training cost | training time: about 20x faster (near 30x on GenEval) | Improved |
| add timestep clip strategy (ablation) | DGPO · visual text rendering | OCR accuracy / visual quality: 0.95 -> 0.96 | Improved |
| use ODE rollout instead of SDE rollout (ablation) | DGPO · convergence and final metrics | reward metrics | Improved |
| online rollouts instead of offline (reference model) data (ablation) | DGPO · reward metrics | reward | Improved |
| use DGPO group objective instead of Diffusion DPO pairwise (ablation) | DGPO · online and offline settings | reward | Improved |
Closest papers by meaning
- Forward KL Regularized Preference Optimization for Aligning Diffusion Policies (AAAI 2025 · similarity 0.90)
- Group Robust Preference Optimization in Reward-free RLHF (NeurIPS 2024 · similarity 0.90)
- Explicit Preference Optimization: No Need for an Implicit Reward Model (ICML 2025 · similarity 0.89)
- Using Human Feedback to Fine-tune Diffusion Models without Any Reward Model (CVPR 2024 · similarity 0.89)
- Exploratory Preference Optimization: Harnessing Implicit Q*-Approximation for Sample-Efficient RLHF (ICLR 2025 · similarity 0.88)
- Ranking-based Preference Optimization for Diffusion Models from Implicit User Feedback (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
