Research map / Image generation / Text-to-image diffusion
Any-Size-Diffusion: Toward Efficient Text-Driven Synthesis for Any-Size HD Images
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Any-Size-Diffusion (ASD), a two-stage pipeline: ARAD fine-tunes Stable Diffusion 2.1 with LoRA on multi-aspect-ratio images for better composition, and FSTD is a training-free tiled super-resolution stage using random-offset implicit overlap to produce high-resolution images of arbitrary size.
Problem
Stable Diffusion produces poor compositions (improper cropping, unnatural layouts) when generating images at resolutions other than its single training scale, and generating very high-resolution images is memory- and time-intensive.
Approach
Fine-tune SD on a restricted set of predefined aspect ratios (ARAD) so composition adapts to varying sizes, then magnify with a tiled sampler that uses random per-step tile offsets as implicit overlap (FSTD), avoiding both seams and the 2x cost of explicit overlapping tiles.
Key results
- On MM-CelebA-HQ, the full ASD reaches FID 85.34, IS 4.04, CLIP 29.23, versus 118.83 FID for the SD 2.1 baseline (SR-Plus).
- ASD reaches 18432^2 max resolution on a 32G GPU, at least 9x the 2048^2 limit of SD 2.1.
- Implicit overlap tiled sampling cuts per-image inference time from 166.8s (explicit overlap) to about 75s, roughly matching non-overlap tiling (75.08s) while removing seams.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| fine-tune SD 2.1 with multi-aspect ratio training | SD 2.1 / ARAD · MA-LAION-COCO | FID: 14.32 -> 13.98 | Helped |
| fine-tune SD 2.1 with multi-aspect ratio training | SD 2.1 / ARAD · MA-COCO | FID: 42.50 -> 40.28 | Helped |
| add FSTD with implicit overlap to AR-Tile | ARAD · MM-CelebA-HQ | FID: 85.66 -> 85.34 | Helped |
| use explicit tile overlap instead of no overlap (ablation) | FSTD tiled sampling · MM-CelebA-HQ | PSNR: 26.89 -> 27.49 | Helped |
| use explicit tile overlap instead of no overlap (ablation) | FSTD tiled sampling · MM-CelebA-HQ | Time per frame: 75.08s -> 166.8s | Hurt |
| use implicit overlap with random offsets instead of explicit overlap (ablation) | FSTD tiled sampling · MM-CelebA-HQ | Time per frame: 166.8s -> 75.19s | Helped |
| use implicit overlap with random offsets instead of fixed offsets (ablation) | FSTD tiled sampling · MM-CelebA-HQ | PSNR: 26.83 -> 27.53 | Helped |
| increase training aspect-ratio set from 3 to 5 (ablation) | ARAD · MA-LAION-COCO | FID: 14.36 -> 14.10 | Helped |
| increase training aspect-ratio set from 5 to all 9 (ablation) | ARAD · MA-LAION-COCO | FID: 14.10 -> 13.98 | Helped |
| vary random offset range among 16, 32, and 48 (ablation) | FSTD · MM-CelebA-HQ | PSNR: 27.51, 27.53, and 27.52 | No clear effect |
| increase explicit tile overlap from 0 to 16 (ablation) | FSTD tiled sampling · MM-CelebA-HQ | PSNR: 26.89 -> 27.50 | Helped |
| increase explicit tile overlap from 0 to 16 (ablation) | FSTD tiled sampling · MM-CelebA-HQ | Time per image: 75.1s -> 148.7s | Hurt |
Surprising findings
- Non-overlapping tiling slightly improves FID over the untiled AR-Plus baseline in the MM-CelebA-HQ table (92.80 to 85.66), which the authors attribute to the tiled algorithm rather than explaining further.
Limitations
- ARAD is restricted to a predefined set of nine aspect ratios and sizes (up to 1024 px), so arbitrary ratios are only approximated.
- Evaluation on MM-CelebA-HQ is limited to face super-resolution, and the high-resolution composition results are mainly qualitative.
- FSTD inherits the cost of StableSR's 200-step DDPM sampling, so inference remains slow (about 75s per image).
Experimental setup
Datasets: LAION-Aesthetic (90k subset) LAION-COCO MS-COCO MM-CelebA-HQ
Benchmarks: MA-LAION-COCO MA-COCO MM-CelebA-HQ
Metrics: FID IS CLIP score PSNR SSIM LPIPS
Baselines: SD 2.1 (SR-Plus) SR-Tile SR-Tile-Plus AR-Plus AR-Tile MultiDiffusion (MD 2.1)
Builds on: Stable Diffusion MultiDiffusion StableSR LoRA
Closest papers by meaning
- Würstchen: An Efficient Architecture for Large-Scale Text-to-Image Diffusion Models (ICLR 2024 · similarity 0.90)
- SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis (ICLR 2024 · similarity 0.90)
- DiffuseHigh: Training-Free Progressive High-Resolution Image Synthesis Through Structure Guidance (AAAI 2025 · similarity 0.90)
- KOALA: Empirical Lessons Toward Memory-Efficient and Fast Diffusion Models for Text-to-Image Synthesis (NeurIPS 2024 · similarity 0.90)
- Simpler Diffusion: 1.5 FID on ImageNet512 with Pixel-space Diffusion (CVPR 2025 · similarity 0.89)
- RealCompo: Balancing Realism and Compositionality Improves Text-to-Image Diffusion Models (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.