Research map / Image generation / Fast diffusion sampling

Adjoint Matching: Fine-tuning Flow and Diffusion Generative Models with Memoryless Stochastic Optimal Control

ICLR 2025 · Image generation › Fast diffusion sampling · approach: Guidance for conditional diffusion · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Casts reward fine-tuning of diffusion and flow matching models as stochastic optimal control, proposes the memoryless noise schedule and the Adjoint Matching loss, and reports gains on text-to-image fine-tuning of Stable Diffusion 3.

Problem

Reward fine-tuning of dynamical generative models (diffusion and flow matching) to sample from a reward-tilted distribution p* ∝ p_base exp(r) in a theoretically sound way.

Approach

Shows via maximum-entropy RL/SOC equivalence that fine-tuning must use a memoryless noise schedule (sigma = sqrt(2 eta_t)) so that the endpoint distribution is tilted by the reward, and introduces the Adjoint Matching loss, a simplification of the continuous adjoint and SOCM-Adjoint losses whose only critical point is the optimal control.

Key results

• Adjoint Matching reaches ImageReward 0.157 vs -0.240 for Continuous Adjoint in multi-prompt fine-tuning with memoryless inference, and ImageReward 1.820 in single-prompt setting.
• On multi-prompt text-to-image, Adjoint Matching gives the best HPS v2 (23.16) and ClipScore/PickScore under flow-matching inference (sigma=0).
• On the simple SOC benchmarks, Adjoint Matching achieves the lowest control L2 error on the 1024-mode bottom-left setting, while SOCM and UW-SOCM perform best in some other settings.

Measured effects reported in the paper

Surprising findings

• Fine-tuning with memoryless noise schedule and then inferring with a deterministic ODE (sigma=0) still improves over the base model, and in some metrics does better than inference with the memoryless schedule.
• ReFL and DRaFT-40 show high variance and instability across metrics in some settings.

Limitations

• Gradient variance of SOCM-type losses blows up with large state and terminal costs and high dimension, so several losses do not scale.
• UW-SOCM shows unstable behavior and weak theoretical guarantees in some settings.
• Fine-tuning is computationally expensive (about 150 seconds per iteration for Adjoint Matching in the reported setup).

Experimental setup

Datasets: fine-tuning text prompt datasets (20000 prompts per run)

Benchmarks: ImageReward ClipScore PickScore HPS v2 DreamSim diversity

Metrics: ImageReward ClipScore PickScore HPS v2 DreamSim diversity control L2 error

Baselines: ReFL DRaFT-1 DRaFT-40 Continuous Adjoint Discrete Adjoint no fine-tuning (base)

Builds on: maximum entropy RL stochastic optimal control matching (SOCM) adjoint ODE flow matching DDPM

Closest papers by meaning

• Efficient Adjoint Matching for Fine-tuning Diffusion Models (NeurIPS 2026 · similarity 0.94)
• Reward Score Matching: Unifying Reward-based Fine-tuning for Flow and Diffusion Models (NeurIPS 2026 · similarity 0.90)
• Online Reward-Weighted Fine-Tuning of Flow Matching with Wasserstein Regularization (ICLR 2025 · similarity 0.90)
• Stochastic Control for Fine-tuning Diffusion Models: Optimality, Regularity, and Convergence (ICML 2025 · similarity 0.89)
• Directly Fine-Tuning Diffusion Models on Differentiable Rewards (ICLR 2024 · similarity 0.89)
• PRECISE: SDE-Consistent Stochastic Sampling for RL Post-Training of Flow-Matching Models (NeurIPS 2026 · similarity 0.88)