Research map / Reinforcement learning / Policy gradient and actor-critic
DiffusionNFT: Online Diffusion Reinforcement with Forward Process
Kaiwen Zheng, Huayu Chen, Haotian Ye, Haoxiang Wang, Qinsheng Zhang, Kai Jiang, Hang Su, Stefano Ermon, Jun Zhu, Ming-Yu Liu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add negative policy loss on v_theta^- (ablation) | DiffusionNFT · online training | reward | Improved |
| use ODE sampler instead of SDE for data collection (ablation) | DiffusionNFT · PickScore | PickScore | Improved |
| use second-order ODE instead of first-order (ablation) | DiffusionNFT · GenEval | GenEval | Improved |
| use second-order ODE instead of first-order (ablation) | DiffusionNFT · PickScore | PickScore | No clear effect |
| adaptive loss weighting instead of w(t)=1-t (ablation) | DiffusionNFT · reward curves | reward | Improved |
| soft EMA update with small then growing eta instead of fully on-policy (eta=0) (ablation) | DiffusionNFT · reward curves | reward | Improved |
| use soft update eta=0.9 instead of small eta (ablation) | DiffusionNFT · convergence speed | reward | Hurt |
| set guidance parameter beta near 1 (ablation) | DiffusionNFT · reward curves | reward | Improved |
| CFG-free DiffusionNFT training versus base without CFG | SD3.5-M · GenEval | GenEval: 0.24 -> 0.94 | Improved |
| DiffusionNFT versus adding CFG at inference | SD3.5-M · GenEval | GenEval: 0.63 -> 0.94 | Improved |
| DiffusionNFT versus FlowGRPO, single-reward GenEval | SD3.5-M · GenEval | GenEval: 0.97 -> 0.98 | Improved |
| DiffusionNFT versus FlowGRPO, PickScore-trained | SD3.5-M · PickScore | PickScore: 23.62 -> 24.03 | Improved |
Later papers that compared with this method
5 later accepted papers report DiffusionNFT as a baseline in their results tables; 5 of them report a better number on most of the results they share with it.
- On-Policy Diffusion Reinforcement Learning Meets Off-Policy Quality Anchoring (ECCV 2026 · better on 8 of 8 shared results)
- Rethinking the Design Space of Reinforcement Learning for Diffusion Models: On the Importance of Likelihood Estimation Beyond Loss Design (ICML 2026 · better on 5 of 6 shared results)
- Flow-DPPO: Divergence Proximal Policy Optimization for Flow Matching Models (NeurIPS 2026 · better on 6 of 6 shared results)
- Embedding-perturbed Exploration Preference Optimization for Flow Models (ICML 2026 · better on 3 of 3 shared results)
- Distillation Models are Good Samplers for Diffusion Reinforcement Learning (ICML 2026 · better on 3 of 3 shared results)
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Designing Reinforcement Learning for Diffusion Models: A Unified Path-Space View (NeurIPS 2026 · similarity 0.91)
- Prior-Guided Diffusion Planning for Offline Reinforcement Learning (NeurIPS 2025 · similarity 0.91)
- Flow-GRPO: Training Flow Matching Models via Online RL (NeurIPS 2025 · similarity 0.90)
- Diffusion Guidance Is a Controllable Policy Improvement Operator (NeurIPS 2026 · similarity 0.90)
- Reinforcement Learning for Diffusion LLMs with Entropy-Guided Step Selection and Stepwise Advantages (NeurIPS 2026 · similarity 0.89)
- Score Regularized Policy Optimization through Diffusion Behavior (ICLR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
