Research map / Image generation / Super-resolution
Latent Space Super-Resolution for Higher-Resolution Image Generation with Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LSRNA, a training-light latent-space super-resolution module (LSR, LIIF-based) plus region-wise noise addition (RNA) guided by Canny edges, to improve reference-based higher-resolution (>1K) diffusion generation without progressive upscaling.
Problem
Diffusion models trained at fixed resolutions (e.g., SDXL at 1024²) produce structural distortions and content repetition when generating larger images; reference-based training-free methods suffer from manifold deviation when upsampling latents and over-smoothing when upsampling in RGB space.
Approach
Train a lightweight LIIF-style latent super-resolution module that maps low-resolution SDXL latents onto the higher-resolution latent manifold, then add edge-modulated (Canny) Gaussian noise to high-frequency regions, enabling latent upsampling without progressive upscaling and with fewer denoising steps.
Key results
- LSRNA-DemoFusion reaches 4096×4096 pFID 29.12 vs DemoFusion 32.89 on OpenImages-Test, with runtime 506 s vs 1507 s.
- LSRNA-Pixelsmith at 4096×4096 achieves FID 84.19 and pFID 29.62 with 313 s, improving over Pixelsmith (FID 84.75, pFID 32.34, 505 s).
- On the 9× ablation, LSR cuts DemoFusion time from 660 s to 272 s with pFID 37.10 (w/o RNA) and 35.95 (full) vs 38.75 for DemoFusion.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Integrate LSRNA into DemoFusion | DemoFusion · OpenImages-Test 2048×2048 | FID: 85.02 -> 83.58 | Helped |
| Integrate LSRNA into DemoFusion | DemoFusion · OpenImages-Test 2048×4096 | pFID: 38.64 -> 33.31 | Helped |
| Integrate LSRNA into DemoFusion | DemoFusion · OpenImages-Test 4096×4096 | Time (sec): 1507 -> 506 | Helped |
| Integrate LSRNA into Pixelsmith | Pixelsmith · OpenImages-Test 2048×2048 | FID: 85.40 -> 83.90 | Helped |
| Integrate LSRNA into Pixelsmith | Pixelsmith · OpenImages-Test 2048×4096 | FID: 81.21 -> 78.47 | Helped |
| Integrate LSRNA into Pixelsmith | Pixelsmith · OpenImages-Test 4096×4096 | FID: 84.75 -> 84.19 | Helped |
| Add RNA to LSRNA-DemoFusion (ablation) | LSRNA-DemoFusion · OpenImages-Valid ×9 | pFID: 37.10 -> 35.95 | Helped |
| Add RNA to LSRNA-DemoFusion (ablation) | LSRNA-DemoFusion · OpenImages-Valid ×9 | FID: 132.65 -> 132.01 | Helped |
| Use RNA with emax 0.8 instead of 1.2 (ablation) | LSRNA-DemoFusion · OpenImages-Valid ×9 | pFID: 35.95 -> 36.40 | Hurt |
| Use RNA with emax 1.0 instead of 1.2 (ablation) | LSRNA-DemoFusion · OpenImages-Valid ×9 | pFID: 35.95 -> 36.12 | Hurt |
| Use RNA with emax 1.4 instead of 1.2 (ablation) | LSRNA-DemoFusion · OpenImages-Valid ×9 | pFID: 35.95 -> 36.04 | Hurt |
Surprising findings
- Integrating LSR alone slightly worsens FID (132.65 vs 131.95) while improving pFID, showing that gains are mostly in patch-level detail metrics.
Limitations
- Optimal RNA strength varies across integrated methods and noise schedules, requiring tuning.
- Generation ability remains bounded by the capacity of the pretrained LDM.
- Quantitative evaluation uses filtered OpenImages subsets (>3K) and captions from BLIP-2.
Experimental setup
Datasets: OpenImages RealSR-style real-world HR images (cited as [28])
Benchmarks: OpenImages-Test
Metrics: FID KID pFID pKID IS CLIP Score inference time
Baselines: SDXL+BSRGAN SDXL (Direct) ScaleCrafter FouriScale HiDiffusion Self-Cascade DemoFusion Pixelsmith
Builds on: LIIF DemoFusion Pixelsmith SDEdit Latent Diffusion Models
Closest papers by meaning
- SinSR: Diffusion-Based Image Super-Resolution in a Single Step (CVPR 2024 · similarity 0.88)
- Simpler Diffusion: 1.5 FID on ImageNet512 with Pixel-space Diffusion (CVPR 2025 · similarity 0.87)
- Noise-Started One-Step Image Super-Resolution via LR-Conditioned SplitMeanFlow and GAN Refinement (NeurIPS 2026 · similarity 0.87)
- Arbitrary-Scale Image Generation and Upsampling using Latent Diffusion Model and Implicit Neural Decoder (CVPR 2024 · similarity 0.87)
- When Latents Forget Pixels: Restoring Fidelity in Diffusion Transformer Super-Resolution (NeurIPS 2026 · similarity 0.87)
- DiffuseHigh: Training-Free Progressive High-Resolution Image Synthesis Through Structure Guidance (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.