Research map / Image generation / Fast diffusion sampling
Fixed Point Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Fixed Point Diffusion Models (FPDM), which embed an implicit fixed-point layer in a DiT-style denoiser trained with stochastic Jacobian-free backpropagation, enabling timestep smoothing and solution reuse for efficient sampling.
Problem
Diffusion models have large parameter counts, high training memory, and expensive sampling, since each sampling timestep requires a full forward pass through a fixed stack of layers.
Approach
Replace the middle explicit transformer layers of a DiT with a single implicit fixed-point layer solved iteratively per timestep, with the solution reused as initialization for the next timestep, and trained via a stochastic multi-step variant of Jacobian-Free Backpropagation.
Key results
- At 280 transformer block forward passes on ImageNet, FPDM achieves FID 43.3 (DDPM) versus DiT's 80.9, with 85M vs 674M parameters.
- FPDM uses 10.2 GB training memory versus 25.2 GB for DiT-XL/2 at batch size 64 (60% less).
- On CelebA-HQ at 280 blocks, FPDM reaches FID 11.1 versus DiT's 65.2.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace DiT with FPDM at 140 transformer block forward passes | ImageNet generation · ImageNet, 140 transformer block forward passes | FID (DDPM): 148.0 -> 85.8 | Helped |
| replace DiT with FPDM at 140 transformer block forward passes | ImageNet generation · ImageNet, 140 transformer block forward passes | FID (DDIM): 110.0 -> 33.9 | Helped |
| replace DiT with FPDM at 280 transformer block forward passes | ImageNet generation · ImageNet, 280 transformer block forward passes | FID (DDPM): 80.9 -> 43.3 | Helped |
| replace DiT with FPDM at 280 transformer block forward passes | ImageNet generation · ImageNet, 280 transformer block forward passes | FID (DDIM): 35.2 -> 22.4 | Helped |
| replace DiT with FPDM at 560 transformer block forward passes | ImageNet generation · ImageNet, 560 transformer block forward passes | FID: DDPM: 37.9 -> 26.1; DDIM: 16.5 -> 19.6 | Mixed |
| replace DiT with FPDM | Image generation · CelebA-HQ, 256px | FID: 65.2 -> 11.1 | Helped |
| replace DiT with FPDM | Image generation · FFHQ, 256px | FID: 58.1 -> 18.2 | Helped |
| replace DiT with FPDM | Image generation · LSUN-Church, 256px | FID: 65.6 -> 22.7 | Helped |
| replace DiT with FPDM | Image generation · ImageNet, 256px, 280 transformer block forward passes | FID: 80.9 -> 43.3 | Helped |
| use 4 to 8 fixed-point iterations per timestep and spread compute across more timesteps (ablation) | FPDM · ImageNet, fixed 280 transformer block forward passes | FID | Helped |
| reuse the previous timestep's fixed-point solution as initialization (ablation) | FPDM · ImageNet generation with few iterations per timestep | FID | Helped |
| use at least one explicit pre/post layer rather than zero (ablation) | FPDM · ImageNet generation | FID | Helped |
Surprising findings
- Solution reuse across timesteps helps most when only a few fixed-point iterations are used per timestep, and is negligible at many iterations.
- The standard 1-step JFB gradient fails badly on ImageNet (FID 567.6), while multi-step and stochastic variants work well.
Limitations
- Performs worse than full DiT when sampling compute and time are unconstrained (DiT has ~8x more parameters).
- Gains stem mainly from smoothing and reuse, which diminish when timesteps and iterations are saturated.
- Only evaluated at 256px on relatively small-scale datasets; scaling to larger datasets left to future work.
Experimental setup
Datasets: ImageNet CelebA-HQ LSUN-Church FFHQ
Benchmarks: ImageNet 256 CelebA-HQ 256 LSUN-Church 256 FFHQ 256
Metrics: FID-50K FID parameter count training memory
Baselines: DiT-XL/2 DiT-B/2 JFB (1-step gradient) Multi-Step JFB
Builds on: Diffusion Transformer (DiT) Deep Equilibrium Models Jacobian-Free Backpropagation Latent Diffusion Models v-prediction
Closest papers by meaning
- Denoising Diffusion Bridge Models (ICLR 2024 · similarity 0.90)
- Diffusion Models Without Attention (CVPR 2024 · similarity 0.89)
- One-step Diffusion with Distribution Matching Distillation (CVPR 2024 · similarity 0.89)
- Diffusion Bridge Implicit Models (ICLR 2025 · similarity 0.89)
- A Simple Early Exiting Framework for Accelerated Sampling in Diffusion Models (ICML 2024 · similarity 0.89)
- FAM Diffusion: Frequency and Attention Modulation for High-Resolution Image Generation with Stable Diffusion (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.