Research map / Image generation / Autoregressive and latent generation
Patched Denoising Diffusion Models For High-Resolution Image Synthesis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Patch-DM, a patch-based diffusion model trained on 64x64 patches that uses feature-level collage of shifted patch features to synthesize seamless high-resolution images without boundary artifacts.
Problem
Diffusion models are hard to scale to high-resolution image synthesis because pixel-space denoising is expensive, and naive patch-based generation produces severe boundary artifacts.
Approach
Instead of shifting patches in pixel space, the UNet encoder features of neighboring patches are split and recombined (feature collage) to form a shifted patch whose noise is predicted by the decoder, so overlapping feature-space context removes border seams; combined with positional embeddings and optimized global semantic conditions, the model stays resolution-agnostic.
Key results
- On the 1024x512 natural image dataset, Patch-DM achieves FID 20.369 vs 44.173 for the best prior patch-based baseline (Anyres-GAN).
- On LSUN-Bedroom 256x256, Patch-DM reaches FID 6.04 vs 15.65 for Anyres-GAN and 10.71 for InfinityGAN among patch-based methods.
- Patch-DM uses 154M parameters for the full model, smaller than ADM (552M), LDM-4 (274M) and DiffAE (232M).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Patch-DM instead of Anyres-GAN | Patch-based image synthesis · Natural images 1024×512 | FID: 44.173 -> 20.369 | Helped |
| use Patch-DM instead of ProjectedGAN | FFHQ image synthesis · FFHQ 256×256 | FID: 3.08 -> 10.02 | Hurt |
| use Patch-DM instead of ADM | LSUN-Bedroom image synthesis · LSUN-Bedroom 256×256 | FID: 1.90 -> 6.04 | Hurt |
| use Patch-DM instead of StyleGAN | LSUN-Church image synthesis · LSUN-Church 256×256 | FID: 4.21 -> 5.49 | Hurt |
| add global semantic condition (ablation) | Patch-DM · Natural images, 1k-image ablation evaluation | FID: 79.33 -> 37.99 | Helped |
| add position embedding (ablation) | Patch-DM · Natural images, 1k-image ablation evaluation | FID: 48.82 -> 37.99 | Helped |
| use feature-level collage instead of fixed pixel-space shift (ablation) | Patch-DM · Natural images, 1k-image ablation evaluation | FID: 49.80 -> 37.99 | Helped |
| use feature-level collage instead of random pixel-space shift (ablation) | Patch-DM · Natural images, 1k-image ablation evaluation | FID: 52.11 -> 37.99 | Helped |
Surprising findings
- Pixel-space collage (fixed or random shifts) still leaves visible border artifacts, with FID around 50 versus 38 for feature collage on the ablation set.
Limitations
- Not all FID comparisons are favorable: on FFHQ, ProjectedGAN and UDM report better FID, and on LSUN-Church LDM-8 and StyleGAN beat Patch-DM.
- Extending outside the training boundary requires post-training to improve continuity, and the 1024x512 setup relies on CLIP-based global conditions optimized per image.
- Evaluation on FID is reported on 50K samples for some settings but 1K for ablations, limiting comparability.
Experimental setup
Datasets: 21,443 self-collected natural images (1024x512) FFHQ LSUN-Bedroom LSUN-Church
Benchmarks: FFHQ 256x256 LSUN-Bedroom 256x256 LSUN-Church 256x256 Natural images 1024x512
Metrics: FID sFID Inception Score Precision Recall
Baselines: COCO-GAN InfinityGAN Anyres-GAN DiffAE LDM-4 ADM UDM IDDPM StyleGAN ProjectedGAN
Builds on: DDPM DDIM classifier-free guidance latent diffusion models CLIP
Closest papers by meaning
- Denoising Diffusion Bridge Models (ICLR 2024 · similarity 0.90)
- DiffuseHigh: Training-Free Progressive High-Resolution Image Synthesis Through Structure Guidance (AAAI 2025 · similarity 0.89)
- Hierarchical Patch Diffusion Models for High-Resolution Video Generation (CVPR 2024 · similarity 0.89)
- ScaleDiff: Higher-Resolution Image Synthesis via Efficient and Model-Agnostic Diffusion (NeurIPS 2025 · similarity 0.89)
- Compressed Image Generation with Denoising Diffusion Codebook Models (ICML 2025 · similarity 0.89)
- Diffusion Bridge Implicit Models (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.