Research map / Image generation / Super-resolution
SEAL: A Framework for Systematic Evaluation of Real-World Super-Resolution
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SEAL, a framework that clusters the real-SR degradation space into 100 representative test cases and evaluates models with acceptance rate (AR) and relative performance ratio (RPR) against acceptance (FSRCNN-mz) and excellence (SRResNet-mz) lines.
Problem
Real-world super-resolution methods are evaluated by average PSNR/LPIPS on small randomly sampled sets of degradations, which fails to represent the vast degradation space and yields inconsistent, misleading rankings.
Approach
Cluster a large sampled degradation space (histogram features, spectral clustering) into K representative degradation cases, define non-blind tiny and large SR models as acceptance and excellence lines, and rank models coarse-to-fine by AR then RPR-based metrics.
Key results
- Under SEAL, BSRNet outperforms RealESRNet on 60% of 100 representative cases (AR 0.55 vs 0.15 on DIV2K_val-SE), reversing the conventional average-PSNR ranking.
- The proposed SwinIR-GD-I reaches AR 0.85 and RPR_I 0.25 on the GAN/MSE comparison table, exceeding BSRNet (AR 0.59).
- Histogram similarity gives clustering purity of 80.2%/80.5% versus 34.6-78.2% for MSE/SSIM.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Evaluate on representative degradation cases with SEAL instead of one randomly generated test set | BSRNet vs RealESRNet · DIV2K_val and DIV2K_val-SE | PSNR: 24.77 vs. 24.93; 24.74 vs. 24.43 | Mixed |
| Use SwinIR as the backbone instead of SRResNet (ablation) | Real-SR model · Set14-SE | AR: 0.12 -> 0.67 | Helped |
| Train on ImageNet instead of DIV2K (ablation) | Real-SR model · Set14-SE | AR: 0.32 -> 0.63 | Helped |
| Set gate probability to 0.75 rather than 1.00 (ablation) | RealESRGAN degradation model · Set14-SE | AR: 0.37 -> 0.44 | Helped |
| Set gate probability to 0.25 rather than 0.75 (ablation) | RealESRGAN degradation model · Set14-SE | AR: 0.44 -> 0.40 | Hurt |
| Use histogram similarity instead of MSE or SSIM for clustering (ablation) | Spectral degradation clustering · Blur100, Noise100, and Blur100 + Noise100 | purity accuracy: Blur100: 78.2\% / 78.2\% -> 80.2\%; Noise100: 39.6\% / 34.6\% -> 80.2\%; Blur100 + Noise100: 51.7\% / 58.2\% -> 80.5\% | Helped |
| Increase sampled degradation count from 5000 to 10000 (ablation) | Degradation-space clustering · Similarity-matrix variance | variance: 8.68 -> 8.71 | No clear effect |
| Increase cluster count toward 100 (ablation) | Spectral degradation clustering · Set14-SE | RPR | No clear effect |
| Use five reference images for clustering instead of one (ablation) | SEAL evaluation of GAN-based real-SR methods · LPIPS | AR: 0.86 vs. 0.75 | Helped |
| Train with gate probability 0.75 rather than 0.25 (ablation) | RealESRGAN degradation model · Set14-SE | RPR_U: 0.21 -> 0.34 | Helped |
| Use RRDBNet instead of SRResNet as the excellence line (ablation) | SEAL evaluation reference line · Real-SR evaluation | performance | Mixed |
Surprising findings
- Real-SR methods tend to perform better on the more challenging degradation cases than on simple ones.
- Conventional random test sets gave inconsistent conclusions: BSRNet and RealESRNet each won in roughly half of 20 random test sets.
Limitations
- Degradation space is built from only two degradation models (BSRGAN and RealESRGAN) and a single reference image (lenna) in the main experiments.
- Acceptance and excellence line choices (FSRCNN-mz, SRResNet-mz) and thresholds are heuristic and user-adjustable, so rankings depend on them.
- Evaluation cost scales linearly with the number of representative cases (inference time 382.74 s vs 4.22 s for Set14).
Experimental setup
Datasets: Set14 DIV2K DIV2K_val ImageNet
Benchmarks: Set14-SE DIV2K_val-SE Set14-Random100 real-DIV2K_val
Metrics: PSNR SSIM LPIPS AR RPR RPR_I RPR_A RPR_U
Baselines: BSRNet RealESRNet SwinIR RealESRNet-GD RDSR DASR ESRGAN BSRGAN MMRealSR RealSRGAN
Builds on: BSRGAN RealESRGAN SRResNet FSRCNN
Closest papers by meaning
- Unsupervised Diffusion-Based Degradation Modeling for Real-World Super-Resolution (AAAI 2025 · similarity 0.89)
- SinSR: Diffusion-Based Image Super-Resolution in a Single Step (CVPR 2024 · similarity 0.88)
- Towards Realistic Data Generation for Real-World Super-Resolution (ICLR 2025 · similarity 0.88)
- Unsupervised Degradation Representation Aware Transform for Real-World Blind Image Super-Resolution (AAAI 2025 · similarity 0.87)
- Low-Res Leads the Way: Improving Generalization for Super-Resolution by Self-Supervised Learning (CVPR 2024 · similarity 0.87)
- StructSR: Refuse Spurious Details in Real-World Image Super-Resolution (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.