Research map / Reinforcement learning / RL from feedback
Half-order Fine-Tuning for Diffusion Model: A Recursive Likelihood Ratio Optimizer
Tao Ren, Zishi Zhang, Jinyang Jiang, Zehao Li, Shentao Qin, Yi Zheng, Guanghao Li, Qianyou Sun, Yan Li, Jiafeng Liang, Xinping Li, Yijie Peng
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use RLR instead of truncated-BP AlignProp | SD1.4 · HPD v2 | ImageReward: 67.18 -> 76.55 | Improved |
| use RLR instead of RL-based DDPO | SD1.4 · HPD v2 | ImageReward: 52.06 -> 76.55 | Improved |
| use RLR instead of VADER for video | VideoCrafter · VBench (6 aspects) | Weighted Average: 83.45 -> 84.63 | Improved |
| add half-order (HO) estimator (ablation) | RLR · HPD v2 | ImageReward: 63.85 -> 76.55 | Improved |
| add zeroth-order (ZO) estimator for remaining steps (ablation) | RLR · HPD v2 | ImageReward: 68.35 -> 76.55 | Improved |
| keep unbiased ZO but drop HO versus keep biased HO but drop ZO (ablation) | RLR · HPD v2 | ImageReward: 63.85 (w/o HO) vs 68.35 (w/o ZO) | Mixed |
| increase sub-chain length h from 0 to 2 (ablation) | RLR · HPD v2 | ImageReward: 63.85 -> 76.55 | Improved |
| increase sub-chain length h beyond 2 (to 4) (ablation) | RLR · HPD v2 | ImageReward: 76.55 -> 76.70 (memory 22.4 -> 45.8 GB) | No clear effect |
| add Diffusive Chain-of-Thought prompts to RLR (ablation) | RLR (SD1.4) · Hand prompts (DCoT) | ImageReward: 43.09 -> 49.88 | Improved |
| add Diffusive Chain-of-Thought prompts to base model (ablation) | SD1.4 · Hand prompts (DCoT) | ImageReward: 33.45 -> 39.24 | Improved |
| swap DDIM sampler for DPMSolver (ablation) | RLR (SD1.4) · Pick-a-Pic | PickScore: 20.14 -> 20.21 | No clear effect |
| use RLR instead of AlignProp for memory | SD1.4 · SD1.4 training memory | total memory: 103.9 GB -> 22.4 GB | Improved |
Closest papers by meaning
- SaRA: High-Efficient Diffusion Model Fine-tuning with Progressive Sparse Low-Rank Adaptation (ICLR 2025 · similarity 0.89)
- Training Diffusion Models Towards Diverse Image Generation with Reinforcement Learning (CVPR 2024 · similarity 0.89)
- Multilevel and Sequential Monte Carlo for Training-Free Diffusion Guidance (NeurIPS 2026 · similarity 0.89)
- Diffusion Tuning: Transferring Diffusion Models via Chain of Forgetting (NeurIPS 2024 · similarity 0.89)
- Towards Better Alignment: Training Diffusion Models with Reinforcement Learning Against Sparse Rewards (CVPR 2025 · similarity 0.89)
- Tuning Timestep-Distilled Diffusion Model Using Pairwise Sample Optimization (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
