Research map / Image generation / Super-resolution
Transformer-Based Selective Super-resolution for Efficient Image Refinement
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Selective Super-Resolution (SSR), a transformer-based method that selects object-containing image tiles via a pyramid tile-selection network and applies deep transformer refinement only to those tiles, reducing computation and improving FID for downstream-oriented SR.
Problem
Transformer-based super-resolution is computationally expensive on whole large images, and refining background high-frequency detail introduces features that degrade downstream computer vision performance.
Approach
Partition the LR image into non-overlapping tiles, use a lightweight Swin-based pyramid tile selector with Gumbel-Softmax hard decisions to classify tiles as object-containing or background, then run a deep residual transformer refinement path only on positive tiles while reconstructing negative tiles directly from shallow features.
Key results
- On BDD100K at 4x, SSR reduces FID from 26.78 (HAT) to 10.41 and KID from 0.0115 to 0.0017.
- SSR reduces computation to 119.40 GMACs versus 191.06 GMACs for HAT on BDD100K at 256x256 input (about 40% reduction).
- Pre-training on ImageNet improves PSNR from 29.56 to 29.91 on BDD100K in the ablation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SSR instead of HAT | Super-resolution · BDD100K, ×4 | FID: 26.78 -> 10.41 | Helped |
| Use SSR instead of HAT | Super-resolution · BDD100K, ×4 | #MACs (G): 191.06 -> 119.40 | Helped |
| Use SSR instead of HAT | Super-resolution · COCO 2017, ×4 | FID: 10.03 -> 2.844 | Helped |
| Use SSR instead of HAT | Super-resolution · MSRA10K, ×4 | FID: 9.577 -> 1.905 | Helped |
| Add ImageNet pre-training (ablation) | SSR · BDD100K | PSNR: 29.56 to 29.91 | Helped |
| Add convolution layers to the Negative Tile Refinement path (ablation) | SSR · BDD100K | FID: 10.41 -> 9.56 | Helped |
| Add convolution layers to the Negative Tile Refinement path (ablation) | SSR · BDD100K | KYD: 0.0041 -> 0.0042 | Hurt |
| Add the TS max block (ablation) | SSR · BDD100K | SSIM: 0.8835 -> 0.8833 | No clear effect |
| Add the TS max block (ablation) | SSR · BDD100K | #Tiles: 77% -> 62% | Hurt |
| Use a pyramid structure for tile selection (ablation) | Tile Selection · Tile selection configurations | maxF | Mixed |
Surprising findings
- On BDD100K, HAT achieves slightly higher PSNR (30.09) than SSR (29.91), so the gains are in perceptual/downstream-oriented metrics rather than PSNR.
- Replacing background pixels with bicubic-upscaled content alone improved FID, suggesting high-frequency background hallucination harms image quality for downstream features.
Limitations
- PSNR is slightly lower than HAT on BDD100K (29.91 vs 30.09).
- Tile labels depend on instance segmentation or detection annotations, limiting applicability to unlabeled data.
- Evaluated on fixed 256x256 LR to 1024x1024 HR setting with 50 epochs of training.
Experimental setup
Datasets: BDD100K COCO 2017 MSRA10K ImageNet
Benchmarks: BDD100K COCO 2017 MSRA10K
Metrics: PSNR SSIM FID KID KYD TPR maxF MACs #Params
Baselines: SwinIR HAT
Builds on: SwinIR Swin Transformer Gumbel-Softmax HAT
Closest papers by meaning
- Recursive Generalization Transformer for Image Super-Resolution (ICLR 2024 · similarity 0.88)
- Transcending the Limit of Local Window: Advanced Super-Resolution Transformer with Adaptive Token Dictionary (CVPR 2024 · similarity 0.87)
- AdaFormer: Efficient Transformer with Adaptive Token Sparsification for Image Super-resolution (AAAI 2024 · similarity 0.86)
- Effective Diffusion Transformer Architecture for Image Super-Resolution (AAAI 2025 · similarity 0.86)
- CFAT: Unleashing Triangular Windows for Image Super-resolution (CVPR 2024 · similarity 0.85)
- Efficient Attention-Sharing Information Distillation Transformer for Lightweight Single Image Super-Resolution (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.