Research map / Architectures and efficiency / Attention and transformers
Efficient Attention-Sharing Information Distillation Transformer for Lightweight Single Image Super-Resolution
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ASID, a lightweight Transformer for single image super-resolution that combines information distillation blocks with attention-sharing across blocks and channel-split, reaching competitive PSNR with about 300K parameters.
Problem
Transformer-based single image super-resolution achieves strong results but its self-attention cost is too high for lightweight, practical deployment.
Approach
Adapts the information distillation scheme to stacked self-attention layers and shares spatial attention matrices across IDBs of the same depth, skipping correlation-matrix computation in later blocks, with channel-split reducing attention channels.
Key results
- ASID-D8 reaches 28.89 dB PSNR on Set14 x4 with 748K parameters, above SwinIR (28.77 dB, 897K).
- ASID achieves 32.88 dB on Urban100 x2 with 298K parameters, versus 32.76 dB for SwinIR at 878K.
- On Set14 x4, ASID uses 313K parameters and 17.2G FLOPs (720p output) versus 49.6G for SwinIR.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add information distillation to the baseline (ablation) | ASID · Urban100 ×2 | PSNR: 32.43 -> 32.70 | Helped |
| Add attention-sharing to information distillation (ablation) | ASID · Urban100 ×2 | PSNR: 32.70 -> 32.62 | Hurt |
| Add channel-split to information distillation (ablation) | ASID · Urban100 ×2 | PSNR: 32.70 -> 32.53 | Hurt |
| Add attention-sharing and channel-split to information distillation (ablation) | ASID · Urban100 ×2 | PSNR: 32.70 -> 32.53 | Hurt |
| Share attention matrices across blocks instead of within a block (ablation) | ASID · Urban100 ×2 | PSNR: 32.49 -> 32.53 | Helped |
Surprising findings
- Sharing attention matrices across blocks of the same depth (InterGroup) outperformed sharing within a block (IntraGroup) while using fewer parameters and FLOPs.
Limitations
- Parameter counts in Urban100 x2 ablation show the sharing gains are modest relative to the baseline
- Comparisons rely on training for 1000 epochs on DIV2K only; generalization beyond reported benchmarks is not analyzed in the main text
Experimental setup
Datasets: DIV2K
Benchmarks: Set5 Set14 B100 Urban100 RealSR
Metrics: PSNR SSIM FLOPs parameters
Baselines: SwinIR ESRT ELAN SPIN Omni-SR IMDN RFDN LatticeNet DRSAN MAFFSRN
Builds on: information distillation network SwinIR ELAN Omni-SR ESRT
Closest papers by meaning
- Transcending the Limit of Local Window: Advanced Super-Resolution Transformer with Adaptive Token Dictionary (CVPR 2024 · similarity 0.89)
- Recursive Generalization Transformer for Image Super-Resolution (ICLR 2024 · similarity 0.87)
- CATANet: Efficient Content-Aware Token Aggregation for Lightweight Image Super-Resolution (CVPR 2025 · similarity 0.87)
- Effective Diffusion Transformer Architecture for Image Super-Resolution (AAAI 2025 · similarity 0.87)
- Progressive Focused Transformer for Single Image Super-Resolution (CVPR 2025 · similarity 0.86)
- Lightweight Image Super-Resolution via Flexible Meta Pruning (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.