Research map / Image generation / Super-resolution
Towards Realistic Data Generation for Real-World Super-Resolution
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RealDGen, an unsupervised framework that decouples content and degradation representations with pretrained extractors to condition a diffusion model, generating large-scale realistic paired LR-HR data for training real-world SR models.
Problem
Real-world super-resolution models generalize poorly because synthetic training degradations (hand-crafted or learned) diverge from real degradations; existing data generators fail to be simultaneously large-scale, realistic, and adaptive to target domains.
Approach
Separately learn a content extractor (VQVAE encoder, reconstruction-trained on LR-HR pairs) and a degradation extractor (contrastive plus reconstruction learning), then condition a Decoupled DDPM on real LR degradation and unpaired HR content, with a filtering step that keeps the lowest-error samples.
Key results
- RRDB trained on RealDGen data reaches 26.238 dB PSNR on RealSR versus 25.488 dB for SynDiff data (+0.75 dB).
- Our generated LR data achieves FID 107.24 on RealSR versus 119.99 for SynDiff, and the user study average score is 8.46/10.
- On out-of-distribution RealVSR with SwinIR-L, PSNR 22.96 vs 22.74 for Real-ESRGAN, outperforming all compared generators.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use RealDGen-generated training data instead of the strongest reported baseline data | SwinIR · RealSR | PSNR: 25.852 -> 26.094 | Helped |
| use RealDGen-generated training data instead of the strongest reported baseline data | SwinIR · DRealSR | PSNR: 28.301 -> 28.721 | Helped |
| use RealDGen-generated training data instead of the strongest reported baseline data | SwinIR · SmartPhone | PSNR: 28.566 -> 28.737 | Helped |
| use RealDGen-generated data for fine-tuning | ResShift · SmartPhone | PSNR: 27.05 -> 27.27 | Helped |
| use RealDGen-generated data for fine-tuning | ResShift · DRealSR | PSNR: 26.19 -> 26.32 | Helped |
| remove degradation extractor E_deg (ablation) | RealDGen with SwinIR-L · SmartPhone | PSNR: 28.868 -> 21.642 | Hurt |
| remove content extractor E_cont (ablation) | RealDGen with SwinIR-L · SmartPhone | PSNR: 28.868 -> 25.745 | Hurt |
| set inference denoising range to 0-200 instead of 0-300 (ablation) | RealDGen · SmartPhone | PSNR: 28.868 -> 28.762 | Hurt |
| set inference denoising range to 0-400 instead of 0-300 (ablation) | RealDGen · SmartPhone | PSNR: 28.868 -> 28.542 | Hurt |
| set inference denoising range to 0-500 instead of 0-300 (ablation) | RealDGen · SmartPhone | PSNR: 28.868 -> 26.713 | Hurt |
| remove generated-sample filtering (ablation) | RealDGen with SwinIR-L · RealSR | PSNR: 26.0250 -> 25.8747 | Hurt |
| freeze extractors during Decoupled DDPM training (ablation) | RealDGen · RealSR | PSNR: 26.16 -> 25.93 | Hurt |
Surprising findings
- Without fine-tuning the extractors, the method still outperforms baselines (25.93 dB vs 25.24 dB for SynDiff on RealSR).
- Fixed denoising steps T=300 is optimal; larger T reduces fidelity, and a lower T fails to generate sufficient degradation.
Limitations
- Diffusion stochasticity sometimes causes loss of fine textures.
- Denoising step count T is manually chosen rather than adapted to degradation level.
- Training and inference are slower than GAN-based hand-crafted degradation pipelines.
Experimental setup
Datasets: DIV2K RealSR DRealSR RealVSR REDS public real LR datasets smartphone-captured LR images
Benchmarks: RealSR DRealSR SmartPhone RealVSR REDS
Metrics: PSNR SSIM LPIPS FID DISTS CLIP-Score CLIP-IQA
Baselines: Real-ESRGAN BSRGAN SynDiff ResShift
Builds on: DDPM VQVAE Real-ESRGAN DASR
Closest papers by meaning
- Unsupervised Diffusion-Based Degradation Modeling for Real-World Super-Resolution (AAAI 2025 · similarity 0.91)
- SinSR: Diffusion-Based Image Super-Resolution in a Single Step (CVPR 2024 · similarity 0.90)
- Noise-Started One-Step Image Super-Resolution via LR-Conditioned SplitMeanFlow and GAN Refinement (NeurIPS 2026 · similarity 0.89)
- Low-Res Leads the Way: Improving Generalization for Super-Resolution by Self-Supervised Learning (CVPR 2024 · similarity 0.89)
- Arbitrary-Scale Image Generation and Upsampling using Latent Diffusion Model and Implicit Neural Decoder (CVPR 2024 · similarity 0.88)
- One-Step Effective Diffusion Network for Real-World Image Super-Resolution (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.