Research map / Image generation / Autoregressive and latent generation
DemoFusion: Democratising High-Resolution Image Generation With No $$$
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DemoFusion, a training-free extension of SDXL that combines progressive upscaling, skip residuals, and dilated sampling over MultiDiffusion to generate 4x-16x higher-resolution images on a single consumer GPU.
Problem
High-resolution image generation with latent diffusion models is expensive to train and centralised; existing LDMs are limited to their training resolution (e.g., 1024² for SDXL) and patch-based methods like MultiDiffusion produce repetitive, semantically incoherent object images.
Approach
A tuning-free plug-and-play framework that builds on MultiDiffusion by progressively upsampling with noise inversion, adding noise-inverted skip residuals for global structure, and using dilated (global) sampling of denoising paths with Gaussian filtering to restore global semantic coherence.
Key results
- At 4096x4096, DemoFusion reaches FIDcrop 70.34 and IScrop 24.28 versus 79.45 / 19.73 for MultiDiffusion and 77.20 / 22.42 for SDXL+BSRGAN, with CLIP 29.57.
- At 2048x2048, DemoFusion achieves FID 65.73 and IS 16.41, the best among compared methods, in 3 min on an RTX 3090.
- Generating 4096x4096 takes about 25 min on a single RTX 3090, longer than the 1-19 min of other training-free baselines (SDXL direct 8 min, SCALECRAFTER 19 min).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DemoFusion instead of SDXL + BSRGAN | SDXL · LAION-5B, 2048 × 2048 | FIDcrop: 67.42 -> 64.81 | Helped |
| Use DemoFusion instead of SDXL + BSRGAN | SDXL · LAION-5B, 2048 × 4096 | FIDcrop: 75.03 -> 71.35 | Helped |
| Use DemoFusion instead of SDXL + BSRGAN | SDXL · LAION-5B, 4096 × 4096 | FIDcrop: 77.20 -> 70.34 | Helped |
| Use DemoFusion instead of SDXL + BSRGAN | SDXL · LAION-5B, 2048 × 2048 | CLIP: 29.61 -> 29.68 | Helped |
| Use DemoFusion instead of SDXL + BSRGAN | SDXL · LAION-5B, 2048 × 4096 | CLIP: 29.01 -> 29.05 | Helped |
| Use DemoFusion instead of SDXL + BSRGAN | SDXL · LAION-5B, 4096 × 4096 | CLIP: 29.63 -> 29.57 | Hurt |
| Add progressive upscaling (ablation) | DemoFusion · High-resolution generation at 3072² | global semantic coherence | Helped |
| Add skip residuals (ablation) | DemoFusion · High-resolution generation at 3072² | global structure consistency | Helped |
| Add dilated sampling (ablation) | DemoFusion · High-resolution generation at 3072² | global semantic coherence | Helped |
Surprising findings
- Directly prompting SDXL at higher resolution fails, yet the model's cropped-photo priors can be exploited to fuse high-resolution patches into coherent scenes.
- Naive dilated sampling produces grainy textures, which the Gaussian filtering and skip residuals are needed to fix.
Limitations
- High computational load from overlapped denoising paths and progressive upscaling leads to long inference times.
- Performance is bounded by the underlying LDM; results on SD 1.5 and SD 2.1 are less impressive than on SDXL.
- Local irrational content can appear in sharp close-up images, and small repetitive background content remains.
Experimental setup
Datasets: LAION-5B
Benchmarks: LAION-5B 1K captions (FID, IS, FIDcrop, IScrop, CLIP score)
Metrics: FID IS FIDcrop IScrop CLIP score inference time
Baselines: SDXL direct inference MultiDiffusion SDXL + BSRGAN SCALECRAFTER
Builds on: MultiDiffusion SDXL latent diffusion models
Closest papers by meaning
- DiffuseHigh: Training-Free Progressive High-Resolution Image Synthesis Through Structure Guidance (AAAI 2025 · similarity 0.90)
- RaPD: Resolution-Agnostic Pixel Diffusion via Semantics-Enriched Implicit Representations (NeurIPS 2026 · similarity 0.88)
- Rethinking Peculiar Images by Diffusion Models: Revealing Local Minima’s Role (AAAI 2024 · similarity 0.88)
- Exploring DCN-like architecture for fast image generation with arbitrary resolution (NeurIPS 2024 · similarity 0.88)
- PaGoDA: Progressive Growing of a One-Step Generator from a Low-Resolution Diffusion Teacher (NeurIPS 2024 · similarity 0.87)
- Spectral Progressive Diffusion for Efficient Image and Video Generation (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.