Research map / Image generation / Super-resolution
RAP-SR: RestorAtion Prior Enhancement in Diffusion Models for Realistic Image Super-Resolution
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds a 5,000-image high-fidelity aesthetic dataset (HFAID) via a multi-stage selection pipeline, fine-tunes Stable Diffusion 2.1 with restoration identifiers (ROPO) and plugs the result into existing diffusion SR methods for Real-SR.
Problem
Pretrained text-to-image diffusion models have suboptimal restoration priors for real-world super-resolution, and manually written restoration prompts (e.g., SeeSR/SUPIR prompts) fail to reliably activate high-quality priors under multiple degradations.
Approach
Curate a small ultra-high-quality aesthetic dataset with a quality/aesthetic selection pipeline, fine-tune the diffusion backbone on it with learned positive/negative restoration identifiers (paired with degraded images as negatives), and use CFG with these identifiers as drop-in replacement for the SR base model without per-method fine-tuning.
Key results
- On DrealSR, SeeSR + RAP-SR raises MANIQA from 0.5941 to 0.6161 and CLIPIQA from 0.6757 to 0.7075 relative to SeeSR alone.
- On RealSR, StableSR + RAP-SR improves CLIPIQA from 0.6315 to 0.6956 and BRISQUE from 19.51 to 15.39.
- Reference metrics are mixed: e.g., on DIV2K, SeeSR + RAP-SR lowers PSNR (23.70 to 23.59) and LPIPS worsens for several methods.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add RAP-SR to StableSR | StableSR · DIV2K val, RealSR, DrealSR | CLIPIQA: 0.6771 -> 0.7517; 0.6315 -> 0.6956; 0.6385 -> 0.6752 | Helped |
| Add RAP-SR to DiffBIR | DiffBIR · DIV2K val, RealSR, DrealSR | CLIPIQA: 0.6693 -> 0.6928; 0.6448 -> 0.6911; 0.6369 -> 0.6729 | Helped |
| Add RAP-SR to SeeSR | SeeSR · DIV2K val, RealSR, DrealSR | CLIPIQA: 0.6936 -> 0.7254; 0.6553 -> 0.7217; 0.6757 -> 0.7075 | Helped |
| Train using 1000 HFAID images instead of the default 5,000 (ablation) | SeeSR + RAP-SR · DrealSR | MANIQA: 0.5681 vs 0.6161 | Hurt |
| Train using 1000 HFAID images instead of the default 5,000 (ablation) | SeeSR + RAP-SR · DrealSR | CLIPIQA: 0.639 vs 0.7075 | Hurt |
| Train using 3000 HFAID images instead of the default 5,000 (ablation) | SeeSR + RAP-SR · DrealSR | MANIQA: 0.5727 vs 0.6161 | Hurt |
| Train using 3000 HFAID images instead of the default 5,000 (ablation) | SeeSR + RAP-SR · DrealSR | CLIPIQA: 0.6423 vs 0.7075 | Hurt |
| Train using 8000 images instead of the default 5,000 (ablation) | SeeSR + RAP-SR · DrealSR | MANIQA: 0.5926 vs 0.6161 | Hurt |
| Train using 8000 images instead of the default 5,000 (ablation) | SeeSR + RAP-SR · DrealSR | CLIPIQA: 0.6866 vs 0.7075 | Hurt |
| Remove positive and negative restoration prompts (ablation) | SeeSR + RAP-SR · DrealSR | MANIQA: 0.5806 vs 0.6161 | Hurt |
| Remove positive and negative restoration prompts (ablation) | SeeSR + RAP-SR · DrealSR | CLIPIQA: 0.6308 vs 0.7075 | Hurt |
| Remove the negative restoration prompt (ablation) | SeeSR + RAP-SR · DrealSR | MANIQA: 0.6063 vs 0.6161 | Hurt |
Surprising findings
- Several full-reference metrics (LPIPS, SSIM, PSNR) worsen after RAP-SR integration on some datasets, which the authors attribute to more realistic details.
- Negative prompts were found to help more than positive prompts in the ablation.
Limitations
- Reference-based metrics often do not improve and sometimes degrade.
- Evaluation relies heavily on no-reference IQA metrics chosen via a small 10-person user study.
- Dataset size ablation shows performance drops beyond 5,000 images, and the dataset is not released.
Experimental setup
Datasets: HFAID LSDIR DIV2K Flickr2K LAION-5B SAM COCO ImageNet
Benchmarks: DIV2K val RealSR DrealSR
Metrics: PSNR SSIM LPIPS MANIQA MUSIQ CLIPIQA BRISQUE
Baselines: StableSR DiffBIR SeeSR SUPIR PASD
Builds on: Stable Diffusion DreamBooth RealESRGAN classifier-free guidance Florence-2
Closest papers by meaning
- TSD-SR: One-Step Diffusion with Target Score Distillation for Real-World Image Super-Resolution (CVPR 2025 · similarity 0.91)
- Taming Diffusion Prior for Image Super-Resolution with Domain Shift SDEs (NeurIPS 2024 · similarity 0.90)
- SinSR: Diffusion-Based Image Super-Resolution in a Single Step (CVPR 2024 · similarity 0.90)
- Noise-Started One-Step Image Super-Resolution via LR-Conditioned SplitMeanFlow and GAN Refinement (NeurIPS 2026 · similarity 0.90)
- One-Step Effective Diffusion Network for Real-World Image Super-Resolution (NeurIPS 2024 · similarity 0.90)
- Solving Diffusion ODEs with Optimal Boundary Conditions for Better Image Super-Resolution (ICLR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.