Research map / Image generation / Text-to-image diffusion
Diffusion-4K: Ultra-High-Resolution Image Synthesis with Latent Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Diffusion-4K, which builds the Aesthetic-4K benchmark (12,015-image 4K training set, 2,781-image eval set) with GLCM Score and Compression Ratio metrics, and fine-tunes SD3-2B and Flux-12B for direct 4096x4096 synthesis via a partitioned VAE and wavelet-based latent fine-tuning (WLF).
Problem
Direct ultra-high-resolution (4K) text-to-image synthesis with latent diffusion models is largely unexplored, and no public 4K training/evaluation benchmark exists; standard metrics (FID, Aesthetics, CLIPScore) are holistic and low-resolution, poorly capturing fine details.
Approach
Combines a partitioned VAE (F=16 via dilated first conv and partitioned decoder output, avoiding retraining) that fits 4K training in memory with a wavelet (Haar DWT) loss on the velocity-prediction objective that emphasizes high-frequency latent components, plus a 4K benchmark with texture-oriented metrics (GLCM Score, JPEG Compression Ratio).
Key results
- WLF on Flux-F16 at 2048 improves FID from 50.57 to 39.49 and CLIPScore from 30.41 to 34.41 on Aesthetic-Eval@2048.
- On Aesthetic-Eval@4096, SD3-F16-WLF improves GLCM Score from 0.73 to 0.77 and reduces Compression Ratio from 11.97 to 10.50 versus the non-finetuned SD3-F16 baseline.
- Partitioned VAEs with F=16 reconstruct 4096x4096 images with PSNR about 28.8 (SD3-VAE) and 29.2 (Flux-VAE) without retraining, and enable inference that OOMs for F=8.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GPT-4o captions instead of LAION-5B captions (ablation) | SD3-F16@4096 · Aesthetic-Eval@4096 | CLIPScore: 29.37 -> 33.12 | Helped |
| Use GPT-4o captions instead of LAION-5B captions (ablation) | SD3-F16@4096 · Aesthetic-Eval@4096 | Aesthetics: 5.90 -> 5.97 | Helped |
| Use GPT-4o captions instead of LAION-5B captions (ablation) | Flux-F16@4096 · Aesthetic-Eval@4096 | CLIPScore: 29.12 -> 33.67 | Helped |
| Use GPT-4o captions instead of LAION-5B captions (ablation) | Flux-F16@4096 · Aesthetic-Eval@4096 | Aesthetics: 6.02 -> 6.11 | Helped |
| Add WLF fine-tuning instead of fine-tuning without WLF (ablation) | SD3-F16@4096 · Aesthetic-Eval@4096 | CLIPScore: 34.14 -> 34.40 | Helped |
| Add WLF fine-tuning instead of fine-tuning without WLF (ablation) | SD3-F16@4096 · Aesthetic-Eval@4096 | Aesthetics: 5.99 -> 6.07 | Helped |
| Add WLF fine-tuning instead of fine-tuning without WLF (ablation) | SD3-F16@4096 · Aesthetic-Eval@4096 | GLCM Score: 0.74 -> 0.77 | Helped |
| Add WLF fine-tuning instead of fine-tuning without WLF (ablation) | SD3-F16@4096 · Aesthetic-Eval@4096 | Compression Ratio: 11.41 -> 10.50 | Helped |
Surprising findings
- Using GPT-4o-generated captions instead of LAION-5B original captions raises CLIPScore by roughly 4-5 points at 4096 resolution.
Limitations
- Evaluation relies on the authors' own metrics (GLCM Score, Compression Ratio) with limited correlation to human judgment (SRCC about 0.36-0.75).
- Gains from WLF over plain fine-tuning are modest (e.g., GLCM 0.74 to 0.77; CLIPScore 34.14 to 34.40).
- Training was on a curated internet image set; the 4K training data is not stated as released.
Experimental setup
Datasets: Aesthetic-Train (12,015 images, GPT-4o captions) LAION-Aesthetics V2 6.5+ (evaluation source)
Benchmarks: Aesthetic-Eval@2048 Aesthetic-Eval@4096
Metrics: FID CLIPScore Aesthetics GLCM Score Compression Ratio rFID PSNR SSIM LPIPS
Baselines: PixArt-Σ Sana Stable Cascade SD3-F8 (original VAE) Flux-F8 (original VAE)
Builds on: Stable Diffusion 3 Flux rectified flow PixArt-Σ
Closest papers by meaning
- SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis (ICLR 2024 · similarity 0.90)
- DiffuseHigh: Training-Free Progressive High-Resolution Image Synthesis Through Structure Guidance (AAAI 2025 · similarity 0.89)
- UltraHR-100K: Enhancing UHR Image Synthesis with A Large-Scale High-Quality Dataset (NeurIPS 2025 · similarity 0.89)
- Simpler Diffusion: 1.5 FID on ImageNet512 with Pixel-space Diffusion (CVPR 2025 · similarity 0.89)
- Any-Size-Diffusion: Toward Efficient Text-Driven Synthesis for Any-Size HD Images (AAAI 2024 · similarity 0.89)
- UltraDiff:Transferring High-Fidelity Priors to Compressed Latent Spaces for High-resolution Image Generation (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.