AI Paper Atlas

Research map / Reinforcement learning / RL from feedback

Half-order Fine-Tuning for Diffusion Model: A Recursive Likelihood Ratio Optimizer

Tao Ren, Zishi Zhang, Jinyang Jiang, Zehao Li, Shentao Qin, Yi Zheng, Guanghao Li, Qianyou Sun, Yan Li, Jiafeng Liang, Xinping Li, Yijie Peng

ICLR 2026 · Reinforcement learning › RL from feedback · approach: RL fine-tuning of generative models · 5 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use RLR instead of truncated-BP AlignPropSD1.4 · HPD v2ImageReward: 67.18 -> 76.55Improved
use RLR instead of RL-based DDPOSD1.4 · HPD v2ImageReward: 52.06 -> 76.55Improved
use RLR instead of VADER for videoVideoCrafter · VBench (6 aspects)Weighted Average: 83.45 -> 84.63Improved
add half-order (HO) estimator (ablation)RLR · HPD v2ImageReward: 63.85 -> 76.55Improved
add zeroth-order (ZO) estimator for remaining steps (ablation)RLR · HPD v2ImageReward: 68.35 -> 76.55Improved
keep unbiased ZO but drop HO versus keep biased HO but drop ZO (ablation)RLR · HPD v2ImageReward: 63.85 (w/o HO) vs 68.35 (w/o ZO)Mixed
increase sub-chain length h from 0 to 2 (ablation)RLR · HPD v2ImageReward: 63.85 -> 76.55Improved
increase sub-chain length h beyond 2 (to 4) (ablation)RLR · HPD v2ImageReward: 76.55 -> 76.70 (memory 22.4 -> 45.8 GB)No clear effect
add Diffusive Chain-of-Thought prompts to RLR (ablation)RLR (SD1.4) · Hand prompts (DCoT)ImageReward: 43.09 -> 49.88Improved
add Diffusive Chain-of-Thought prompts to base model (ablation)SD1.4 · Hand prompts (DCoT)ImageReward: 33.45 -> 39.24Improved
swap DDIM sampler for DPMSolver (ablation)RLR (SD1.4) · Pick-a-PicPickScore: 20.14 -> 20.21No clear effect
use RLR instead of AlignProp for memorySD1.4 · SD1.4 training memorytotal memory: 103.9 GB -> 22.4 GBImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy