Research map / Image generation / Super-resolution
Transcending the Limit of Local Window: Advanced Super-Resolution Transformer with Adaptive Token Dictionary
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ATD, a super-resolution Transformer that uses a learnable, image-adaptively refined token dictionary for cross-attention and category-based self-attention to break window limits.
Problem
Window-based self-attention in SR Transformers limits receptive field, and expanding windows raises computational cost.
Approach
Introduces a learnable token dictionary (inspired by dictionary learning) that provides external prior via cross-attention, is adaptively refined per image via a reverse attention step, and is used to partition tokens into content-based categories for non-local self-attention.
Key results
- ATD achieves 40.35 dB PSNR on Urban100 at x2 (20.1M params), 0.25-0.28 dB above HAT on Urban100.
- ATD-light reaches 31.48 dB on Manga109 x4 with 769K params, 0.46 dB above OmniSR.
- Ablation: TDCA and ADR add 0.11 dB on Urban100; adaptive category attention adds 0.05-0.08 dB over random categorization.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add TDCA and adaptive dictionary refinement (ablation) | ATD-light · Urban100 and Manga109 | PSNR: 26.25 -> 26.36; 30.66 -> 30.79 | Helped |
| Add adaptive category-based MSA to TDCA + ADR (ablation) | ATD-light · Urban100 and Manga109 | PSNR: 26.36 -> 26.51; 30.79 -> 30.98 | Helped |
| Use random category attention instead of no category attention (ablation) | ATD-light · Set5, Urban100, and Manga109 | PSNR: 32.30 -> 32.38; 26.25 -> 26.46; 30.66 -> 30.92 | Helped |
| Use adaptive categorization instead of random categorization (ablation) | ATD-light · Set5, Urban100, and Manga109 | PSNR: 32.38 -> 32.46; 26.46 -> 26.51; 30.92 -> 30.98 | Helped |
| Increase sub-category size ns from 0 to 128 (ablation) | ATD-light · Urban100 | PSNR: 26.36 -> 26.51 | Helped |
| Increase sub-category size ns from 128 to 192 (ablation) | ATD-light · Urban100 and Manga109 | PSNR: 26.51 -> 26.55; 30.98 -> 31.01 | Helped |
| Increase dictionary size M from 16 to 64 (ablation) | ATD-light · Urban100 and Manga109 | PSNR: 26.45 -> 26.51; 30.90 -> 30.98 | Helped |
| Increase dictionary size M from 64 to 96 (ablation) | ATD-light · Urban100 and Manga109 | PSNR: 26.51 -> 26.51; 30.98 -> 30.95 | Hurt |
| Use ATD instead of HAT | classical SR · Urban100 ×4 | PSNR: 27.97 -> 28.22 | Helped |
| Use ATD-light instead of OmniSR | lightweight SR · Manga109 ×4 | PSNR: 31.02 -> 31.48 | Helped |
Limitations
- Ablations are run only on the lightweight model with 250k iterations on DIV2K.
- Category-based attention relies on sub-category splitting with fixed size to ensure parallelism, which limits the per-group receptive field.
Experimental setup
Datasets: DIV2K
Benchmarks: Set5 Set14 BSD100 Urban100 Manga109
Metrics: PSNR SSIM
Baselines: EDSR RCAN SAN HAN IPT SwinIR EDT CAT-A ART HAT CARN IMDN
Builds on: SwinIR dictionary learning for super-resolution HAT ELAN
Closest papers by meaning
- AdaFormer: Efficient Transformer with Adaptive Token Sparsification for Image Super-resolution (AAAI 2024 · similarity 0.90)
- Recursive Generalization Transformer for Image Super-Resolution (ICLR 2024 · similarity 0.90)
- Efficient Attention-Sharing Information Distillation Transformer for Lightweight Single Image Super-Resolution (AAAI 2025 · similarity 0.89)
- CATANet: Efficient Content-Aware Token Aggregation for Lightweight Image Super-Resolution (CVPR 2025 · similarity 0.87)
- Transformer-Based Selective Super-resolution for Efficient Image Refinement (AAAI 2024 · similarity 0.87)
- Effective Diffusion Transformer Architecture for Image Super-Resolution (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.