Research map / Image generation / Super-resolution
Iterative Token Evaluation and Refinement for Real-World Super-resolution
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ITER, a discrete diffusion framework over VQGAN token indices with a learned token evaluation network that selects which tokens to refine and adaptively chooses the starting refinement step for real-world super-resolution.
Problem
Real-world image super-resolution suffers from complex unknown degradations; GAN training is unstable and continuous diffusion models need many sampling steps.
Approach
Separate RWSR into distortion removal (LQ-to-HQ token prediction) and texture generation via a conditioned discrete diffusion model in VQGAN token space, with an added token evaluation network that decides which tokens to keep at each step and sets the number of refinement steps adaptively.
Key results
- On real-world benchmarks, ITER achieves best NIQE and PI on 3 of 4 datasets (e.g., RealSR NIQE 4.67 / PI 4.47; DRealSR NIQE 4.15 / PI 4.27) versus GAN and LDM-BSR baselines.
- ITER runs in 1.7 s per 128x128 to 512x512 image with 8 iterations, versus 4.2 s for LDM-BSR and 0.06 s for RRDBNet on a V100.
- Top-k masking suffers from local propagation, which the learned token evaluation network avoids; smaller threshold alpha gives higher PSNR while alpha=0.5 gives best LPIPS.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ITER instead of MM-RealSR | Real-world super-resolution · RealSR | NIQE: 4.69 -> 4.67 | Helped |
| Use ITER instead of MM-RealSR | Real-world super-resolution · RealSR | PI: 4.50 -> 4.47 | Helped |
| Use ITER instead of FeMaSR | Real-world super-resolution · DRealSR | NIQE: 4.20 -> 4.15 | Helped |
| Use ITER instead of FeMaSR | Real-world super-resolution · DRealSR | PI: 4.30 -> 4.27 | Helped |
| Use ITER instead of SwinIR-GAN | Real-world super-resolution · DPED-iphone | NIQE: 4.95 -> 4.84 | Helped |
| Use ITER instead of FeMaSR | Real-world super-resolution · DPED-iphone | PI: 4.36 -> 4.23 | Helped |
| Use ITER instead of FeMaSR | Real-world super-resolution · RealSRSet | NIQE: 5.18 -> 5.29 | Hurt |
| Use ITER instead of FeMaSR | Real-world super-resolution · RealSRSet | PI: 4.31 -> 4.62 | Hurt |
| Add iterative token refinement after distortion removal (ablation) | ITER · Synthetic DIV2K validation set | LPIPS | Helped |
| Add the token evaluation network instead of top-k confidence masking (ablation) | ITER · Iterative texture generation | texture consistency | Helped |
| Increase threshold α (ablation) | ITER adaptive inference · Synthetic DIV2K validation set | PSNR | Hurt |
| Set threshold α to 0.5 (ablation) | ITER adaptive inference · Synthetic DIV2K validation set | LPIPS | Helped |
Surprising findings
- Token prediction alone causes color shifts, which the authors correct with a test-time color correction step aligning LQ and SR statistics.
Limitations
- Upper bound limited by VQGAN reconstruction quality (0.088 LPIPS), losing fine details such as small humans.
- Inference is slower than pure convolutional RRDBNet and SwinIR due to Swin-based token networks.
- Training of E_l, phi_e and phi_r was done concurrently rather than in the theoretically optimal sequence.
Experimental setup
Datasets: DIV2K Flickr2K OutdoorSceneTraining RealSR DRealSR DPED-iphone RealSRSet
Benchmarks: RealSR DRealSR DPED-iphone RealSRSet DIV2K validation (synthetic)
Metrics: NIQE PI PSNR LPIPS
Baselines: BSRGAN Real-ESRGAN SwinIR-GAN FeMaSR MM-RealSR LDM-BSR
Builds on: VQGAN discrete diffusion model (MaskGIT-style masking) Real-ESRGAN SwinIR
Closest papers by meaning
- Noise-Started One-Step Image Super-Resolution via LR-Conditioned SplitMeanFlow and GAN Refinement (NeurIPS 2026 · similarity 0.90)
- Adversarial Diffusion Compression for Real-World Image Super-Resolution (CVPR 2025 · similarity 0.88)
- Unsupervised Diffusion-Based Degradation Modeling for Real-World Super-Resolution (AAAI 2025 · similarity 0.88)
- Towards Realistic Data Generation for Real-World Super-Resolution (ICLR 2025 · similarity 0.88)
- AdaFormer: Efficient Transformer with Adaptive Token Sparsification for Image Super-resolution (AAAI 2024 · similarity 0.87)
- TSD-SR: One-Step Diffusion with Target Score Distillation for Real-World Image Super-Resolution (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.