Research map / Image generation / Image restoration
Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wild
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Scales a generative-prior image restoration model (SUPIR) built on SDXL with a trimmed ControlNet-style adaptor, ZeroSFT connector, 20M-image text-annotated dataset, MLLM captions, negative-quality prompts, and restoration-guided sampling.
Problem
Image restoration has not benefited from model scaling; generative-prior IR methods are limited by generative model scale, and scaling requires architecture, data, and compute solutions while keeping outputs faithful to low-quality inputs.
Approach
Use SDXL base (2.6B) as the generative prior with a trimmed ControlNet adaptor and a ZeroSFT connector (zero convolution plus spatial feature transfer), train on 20M high-quality captioned images, use LLaVA captions and negative-quality prompts with CFG, and apply restoration-guided EDM sampling to balance fidelity and quality.
Key results
- On real-world images, SUPIR gets CLIP-IQA 0.8232, MUSIQ 73.00 and MANIQA 0.4295, the best on all three non-reference metrics versus PASD (0.7714, 71.87, 0.3169).
- In a 20-participant user study on real-world images, 84.17% of choices favored SUPIR over StableSR, DiffBIR and PASD.
- On synthetic SR x4, SUPIR gets the best ManIQA, ClipIQA and MUSIQ but lower PSNR/SSIM than BSRGAN and PASD, so full-reference metrics disagree with visual quality.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SUPIR instead of the strongest reported baseline | SUPIR · SR (×4) | MUSIQ: 72.94 -> 73.83 | Helped |
| Use SUPIR instead of the strongest reported baseline | SUPIR · SR (×4) | PSNR: 25.06 -> 22.66 | Hurt |
| Use SUPIR instead of the strongest reported baseline | SUPIR · SR (×8) | MUSIQ: 71.74 -> 73.16 | Helped |
| Use SUPIR instead of the strongest reported baseline | SUPIR · SR (×8) | PSNR: 22.26 -> 20.68 | Hurt |
| Use SUPIR instead of the strongest reported baseline | SUPIR · Blur (σ=2) + SR (×4) | MUSIQ: 72.87 -> 73.55 | Helped |
| Use SUPIR instead of the strongest reported baseline | SUPIR · Blur (σ=2) + SR (×4) | PSNR: 24.97 -> 22.43 | Hurt |
| Use SUPIR instead of the strongest reported baseline | SUPIR · SR (×4) + Noise (σ=40) | MUSIQ: 69.09 -> 73.35 | Helped |
| Use SUPIR instead of the strongest reported baseline | SUPIR · SR (×4) + Noise (σ=40) | PSNR: 22.08 -> 20.77 | Hurt |
| Use SUPIR instead of the strongest reported baseline | SUPIR · Blur (σ=2) + SR (×4) + Noise (σ=20) + JPEG (q=50) | MUSIQ: 70.94 -> 73.58 | Helped |
| Use SUPIR instead of the strongest reported baseline | SUPIR · Blur (σ=2) + SR (×4) + Noise (σ=20) + JPEG (q=50) | PSNR: 22.88 -> 20.84 | Hurt |
| Replace ZeroSFT with zero convolution (ablation) | SUPIR · Real-world LQ images | PSNR: 20.66 -> 19.47 | Hurt |
| Replace ZeroSFT with zero convolution (ablation) | SUPIR · Real-world LQ images | LPIPS: 0.3412 -> 0.3969 | Hurt |
Surprising findings
- Full-reference metrics (PSNR, SSIM, LPIPS) favor methods with lower perceptual quality, so SUPIR scores lower on them despite better visual results.
- Adding low-quality SDXL-generated negative samples to training was needed for negative-quality prompts to help rather than introduce artifacts.
Limitations
- Full-reference metrics do not reflect perceptual quality, so evaluation relies partly on user study.
- Text prompts are ineffective when they conflict with the low-quality input.
- Restoration-guided sampling trades off fidelity and detail via tau_r, and very small values blur output.
Experimental setup
Datasets: LSDIR DIV2K FFHQ-raw RealSR DRealSR Real47
Benchmarks: synthetic degradation test sets (SR x4, SR x8, blur+SR, noise+SR, JPEG mixtures) 60 real-world LQ images user study with 20 participants
Metrics: PSNR SSIM LPIPS ManIQA ClipIQA MUSIQ user preference
Baselines: BSRGAN Real-ESRGAN StableSR DiffBIR PASD
Builds on: SDXL ControlNet EDM sampler Real-ESRGAN degradation model DiffBIR
Closest papers by meaning
- RAP-SR: RestorAtion Prior Enhancement in Diffusion Models for Realistic Image Super-Resolution (AAAI 2025 · similarity 0.89)
- UP-Restorer: When Unrolling Meets Prompts for Unified Image Restoration (AAAI 2025 · similarity 0.88)
- UARE: A Unified Vision-Language Model for Image Quality Assessment, Restoration, and Enhancement (CVPR 2026 · similarity 0.88)
- ReFIR: Grounding Large Restoration Models with Retrieval Augmentation (NeurIPS 2024 · similarity 0.88)
- DreamClear: High-Capacity Real-World Image Restoration with Privacy-Safe Dataset Curation (NeurIPS 2024 · similarity 0.88)
- The Power of Context: How Multimodality Improves Image Super-Resolution (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.