Research map / Image generation / Fast diffusion sampling
AdjointDEIS: Efficient Gradients for Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AdjointDEIS, a family of exponential-integrator solvers (k=1,2) for continuous adjoint equations of diffusion ODEs and SDEs, showing SDE adjoints reduce to an ODE, and applies it to face morphing attacks.
Problem
Computing gradients of a differentiable guidance loss on diffusion model outputs with respect to latents, conditioning, and parameters requires backpropagating through the whole ODE/SDE sampler, which is memory intensive and awkward for SDE samplers.
Approach
Derive exponential-integrator solutions of the continuous adjoint equations in log-SNR domain, with Taylor expansion and analytic phi-function coefficients (AdjointDEIS-k, incl. multistep 2M), and show that for VP diffusion SDEs with scalar diffusion coefficient the adjoint equations reduce to an ODE.
Key results
- DiM + AdjointDEIS-1 (ODE) reaches MMPMR of 99.8% on AdaFace, 98.77% on ArcFace, and 99.39% on ElasticFace on SYN-MAD 2022, above all baselines including Webmorph (97.96/96.93/98.36).
- DiM + AdjointDEIS-1 (SDE) achieves 98.57/97.96/97.75 MMPMR, lower than the ODE variant.
- The paper proves AdjointDEIS-k is a k-th order solver (k=1,2) with global truncation error O(h^2) for the adjoint state.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use AdjointDEIS-1 with ODE sampling instead of DiM-A | DiM face morphing · SYN-MAD 2022, AdaFace | MMPMR: 92.23 -> 99.8 | Helped |
| Use AdjointDEIS-1 with ODE sampling instead of DiM-A | DiM face morphing · SYN-MAD 2022, ArcFace | MMPMR: 90.18 -> 98.77 | Helped |
| Use AdjointDEIS-1 with ODE sampling instead of DiM-A | DiM face morphing · SYN-MAD 2022, ElasticFace | MMPMR: 93.05 -> 99.39 | Helped |
| Use AdjointDEIS-1 with SDE sampling instead of DiM-A | DiM face morphing · SYN-MAD 2022, AdaFace | MMPMR: 92.23 -> 98.57 | Helped |
| Use AdjointDEIS-1 with SDE sampling instead of DiM-A | DiM face morphing · SYN-MAD 2022, ArcFace | MMPMR: 90.18 -> 97.96 | Helped |
| Use AdjointDEIS-1 with SDE sampling instead of DiM-A | DiM face morphing · SYN-MAD 2022, ElasticFace | MMPMR: 93.05 -> 97.75 | Helped |
| Use SDE rather than ODE sampling with AdjointDEIS-1 | DiM face morphing · SYN-MAD 2022, AdaFace | MMPMR: 99.8 -> 98.57 | Hurt |
| Use SDE rather than ODE sampling with AdjointDEIS-1 | DiM face morphing · SYN-MAD 2022, ArcFace | MMPMR: 98.77 -> 97.96 | Hurt |
| Use SDE rather than ODE sampling with AdjointDEIS-1 | DiM face morphing · SYN-MAD 2022, ElasticFace | MMPMR: 99.39 -> 97.75 | Hurt |
Surprising findings
- The SDE variant of AdjointDEIS underperforms the ODE variant, which the authors attribute to difficulty discretizing SDEs with large step sizes.
- Adjoint of diffusion SDEs with scalar diffusion coefficient evolves by an ODE despite the forward process being stochastic.
Limitations
- Evaluated on only a single scenario (face morphing).
- Limited hyperparameter exploration (learning rate, steps).
- Solvers only constructed and analyzed up to order k=2; higher-order solvers may be unsuitable for large guidance scales.
Experimental setup
Datasets: FFHQ SYN-MAD 2022 Face Research Lab London (FRLL) dataset
Benchmarks: SYN-MAD 2022
Metrics: MMPMR at FMR=0.1% NFE
Baselines: DiM-A Fast-DiM Morph-PIPE MIPGAN-I MIPGAN-II Webmorph
Builds on: DEIS Adjoint sensitivity method for neural ODEs Diffusion Morph (DiM) CycleDiffusion
Closest papers by meaning
- Adjoint Sampling: Highly Scalable Diffusion Samplers via Adjoint Matching (ICML 2025 · similarity 0.91)
- AdjointDPM: Adjoint Sensitivity Method for Gradient Backpropagation of Diffusion Probabilistic Models (ICLR 2024 · similarity 0.90)
- S4S: Solving for a Fast Diffusion Model Solver (ICML 2025 · similarity 0.88)
- Non-equilibrium Annealed Adjoint Sampler (NeurIPS 2025 · similarity 0.88)
- Align Your Steps: Optimizing Sampling Schedules in Diffusion Models (ICML 2024 · similarity 0.88)
- Efficient Adjoint Matching for Fine-tuning Diffusion Models (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.