Research map / Image generation / Text-to-image diffusion

Efficient Adjoint Matching for Fine-tuning Diffusion Models

NeurIPS 2026 · Image generation › Text-to-image diffusion · approach: Preference optimization for diffusion · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Efficient Adjoint Matching (EAM), which reformulates the SOC-based reward fine-tuning of diffusion/flow models with a linear base drift and corrected terminal cost so that trajectories come from few-step ODE plus closed-form noising and the adjoint has a closed form, yielding up to 4x faster training than Adjoint Matching on SD3.5-Medium.

Problem

Adjoint Matching for reward-gradient fine-tuning of diffusion/flow models is computationally expensive due to stochastic forward SDE simulation with many NFEs and backward adjoint ODE simulation requiring JVPs through the pretrained network.

Approach

Replace the non-trivial pretrained base drift with a linear base drift D(t)x chosen so the base noising kernel matches the original linear-path kernel and the base is memoryless; this gives a closed-form adjoint (scalar scaling of the endpoint reward gradient) and allows sampling X_t directly from X_1, with a corrected terminal cost that still targets the reward-tilted distribution.

Key results

• EAM reaches PickScore 23.16 vs AM 23.00 on DrawBench under the PickScore-only reward, with similar or better scores on HPSv2.1 (0.297 vs 0.292) and Aesthetics (6.27 vs 6.25).
• Under the multi-reward setting, EAM attains ImageReward 0.98 and Aesthetics 6.56 versus AM's 0.83 and 6.21, while PickScore is slightly lower (22.93 vs 23.06).
• EAM reduces training time by up to 4x relative to AM on 4 A100 GPUs, using 10 NFEs instead of 40 for trajectory sampling and no backward adjoint simulation.

Surprising findings

• EAM has lower CLIPScore than AM in both settings (0.252 vs 0.259 single-reward), so improved preference metrics do not uniformly improve prompt alignment.
• Applying CFG to AM and EAM improves most metrics but reduces Aesthetics, and doubles NFEs.

Limitations

• Practical optimization strategies such as buffer replay, old-policy trajectory generation, and adaptive hyperparameter schedules are left for future work.
• Multi-reward coefficients and reward scales were tuned separately for AM and EAM, and the hyperparameter C required tuning.
• Evaluation limited to one base model (SD3.5-M) and one prompt benchmark (DrawBench).

Experimental setup

Datasets: Pick-a-Pic

Benchmarks: DrawBench

Metrics: PickScore CLIPScore ImageReward HPSv2.1 Aesthetics

Baselines: Adjoint Matching SD3.5-M + CFG ASBS TR-SOCM DiffusionNFT

Builds on: Adjoint Matching Adjoint Sampling DPM-Solver++ Classifier-Free Guidance DiffusionNFT

Closest papers by meaning

• Adjoint Matching: Fine-tuning Flow and Diffusion Generative Models with Memoryless Stochastic Optimal Control (ICLR 2025 · similarity 0.94)
• Online Reward-Weighted Fine-Tuning of Flow Matching with Wasserstein Regularization (ICLR 2025 · similarity 0.89)
• Directly Fine-Tuning Diffusion Models on Differentiable Rewards (ICLR 2024 · similarity 0.89)
• Reward Score Matching: Unifying Reward-based Fine-tuning for Flow and Diffusion Models (NeurIPS 2026 · similarity 0.88)
• TMPO: Trajectory Matching Policy Optimization for Diverse and Efficient Diffusion Alignment (NeurIPS 2026 · similarity 0.88)
• AdjointDPM: Adjoint Sensitivity Method for Gradient Backpropagation of Diffusion Probabilistic Models (ICLR 2024 · similarity 0.88)