Research map / Image generation / Autoregressive and latent generation
Relay Diffusion: Unifying diffusion process across resolutions for image synthesis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Relay Diffusion Model (RDM), a cascaded framework where the high-resolution stage starts diffusion from the upsampled low-resolution result using block noise and patch-wise blurring diffusion, achieving strong FID on CelebA-HQ and ImageNet 256x256.
Problem
Diffusion models struggle to train end-to-end at high resolution because the same noise level yields higher frequency-domain SNR at higher resolution, making noise schedules resolution-dependent; cascaded pipelines are costly and suffer from conditioning mismatch.
Approach
Relay diffusion continues the diffusion process across resolutions: the low-resolution output is upsampled and corrupted with block noise (equivalent to low-res Gaussian noise at high-res) and patch-wise blurring, so the high-res stage starts from the low-res result rather than pure noise or low-res conditioning; a matching second-order stochastic sampler is derived.
Key results
- RDM achieves FID 3.15 on unconditional CelebA-HQ 256x256, beating StyleSwin (3.25) and LDM-4 (5.11) with fewer training images (50M vs 820M).
- On class-conditional ImageNet 256x256 with CFG, RDM reaches FID 1.87 (with class-balance) and the best sFID of 3.97, though MDT-XL/2 with dynamic CFG attains a lower FID of 1.79.
- Without guidance RDM (FID 5.27) beats DiT-XL/2 and MDT-XL/2 (FID 9.62 and 6.23) while using about one-third of their training iterations.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use RDM instead of StyleSwin | CelebA-HQ 256×256 generation · CelebA-HQ 256×256 | FID: 3.25 -> 3.15 | Helped |
| use RDM instead of MDT-XL/2 | ImageNet class-conditional generation without CFG · ImageNet 256×256 | FID: 6.23 -> 5.27 | Helped |
| use RDM instead of ADM | ImageNet class-conditional generation without CFG · ImageNet 256×256 | sFID: 6.02 -> 4.39 | Helped |
| add class-balance (ablation) | RDM with CFG=3.50 · ImageNet 256×256 | FID: 1.99 -> 1.87 | Helped |
| add block noise (α=0.15, kernel size s=4) (ablation) | RDM · ImageNet 256×256 and CelebA-HQ 256×256 | FID | Helped |
| add block noise (α=0.15, kernel size s=4) (ablation) | RDM · ImageNet 256×256 | training convergence | Hurt |
| set sampler stochasticity η=0.10 instead of η=0 (ablation) | RDM · ImageNet 256×256 | FID: 5.65 -> 5.44 | Helped |
| set sampler stochasticity η=0.15 instead of η=0 (ablation) | RDM · ImageNet 256×256 | FID: 5.65 -> 5.31 | Helped |
| set sampler stochasticity η=0.20 instead of η=0 (ablation) | RDM · ImageNet 256×256 | FID: 5.65 -> 5.27 | Helped |
| set sampler stochasticity η=0.50 instead of η=0 (ablation) | RDM · ImageNet 256×256 | FID: 5.65 -> 9.17 | Hurt |
| set sampler stochasticity η=0.20 instead of η=0 (ablation) | RDM · CelebA-HQ 256×256 | FID: 4.11 -> 3.15 | Helped |
| set sampler stochasticity η=0.50 instead of η=0 (ablation) | RDM · CelebA-HQ 256×256 | FID: 4.11 -> 6.41 | Hurt |
Surprising findings
- Block noise slows early training convergence on ImageNet but yields a clearly better final FID after sufficient training.
- The authors' attempts to directly derive an optimal noise schedule from the dataset did not yield good results.
Limitations
- Optimal noise schedule could not be derived directly from data; schedule choice remains heuristic.
- On ImageNet, RDM does not reach the best FID, which MDT-XL/2 with dynamic CFG achieves; longer training and finer CFG are left as future work.
- Block noise slows early convergence, and the weighting-average strategy is a heuristic rather than the ideal gradual transition.
Experimental setup
Datasets: ImageNet CelebA-HQ
Benchmarks: CelebA-HQ 256x256 ImageNet 256x256
Metrics: FID sFID IS Precision Recall
Baselines: ADM LDM-4 DiT-XL/2 MDT-XL/2 StyleGAN-XL BigGAN-deep StyleSwin LSGM WaveDiff CDM
Builds on: EDM ADM Cascaded diffusion models Blurring diffusion models DDIM
Closest papers by meaning
- FAM Diffusion: Frequency and Attention Modulation for High-Resolution Image Generation with Stable Diffusion (CVPR 2025 · similarity 0.90)
- DiffuseHigh: Training-Free Progressive High-Resolution Image Synthesis Through Structure Guidance (AAAI 2025 · similarity 0.90)
- Arbitrary-steps Image Super-resolution via Diffusion Inversion (CVPR 2025 · similarity 0.89)
- Taming Diffusion Prior for Image Super-Resolution with Domain Shift SDEs (NeurIPS 2024 · similarity 0.89)
- A Simple Early Exiting Framework for Accelerated Sampling in Diffusion Models (ICML 2024 · similarity 0.89)
- Simpler Diffusion: 1.5 FID on ImageNet512 with Pixel-space Diffusion (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.