Research map / Image generation / Super-resolution
CATANet: Efficient Content-Aware Token Aggregation for Lightweight Image Super-Resolution
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CATANet, a lightweight image super-resolution transformer that aggregates long-range content-similar tokens via shared, training-only-updated token centers, followed by intra-group self-attention and inter-group cross-attention.
Problem
Transformer-based image super-resolution relies on content-agnostic local windows, which limits long-range dependency capture, while existing clustering-based methods like SPIN are slow at inference, which hinders lightweight deployment.
Approach
A Content-Aware Token Aggregation module shares EMA-updated token centers across all images and updates them only during training, so content-similar tokens are grouped at inference with no clustering iterations; intra-group self-attention operates on these groups (with subgroup overlap to balance sizes) and inter-group cross-attention attends to the token centers.
Key results
- Outperforms SPIN on all five benchmarks at x2/x3/x4 with a maximum PSNR gain of 0.33 dB without self-ensemble (0.60 dB with self-ensemble) and 15K fewer parameters.
- Inference time of CATANet-L is 86 ms versus 435 ms for SPIN on a RTX 4090 (about 5x faster) and roughly 2x faster than SwinIR-light.
- Subgrouping of token groups roughly doubles inference speed (86 ms vs 188 ms without subgrouping).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use CATANet instead of SPIN | lightweight image SR · Urban100 ×4 | PSNR: 26.55 -> 26.87 | Helped |
| add IASA to the LRSA-only baseline (ablation) | CATANet · Urban100 ×4 | PSNR: 26.46 -> 26.85 | Helped |
| add IRCA to IASA (ablation) | CATANet · Urban100 ×4 | PSNR: 26.85 -> 26.87 | Helped |
| allow each subgroup to attend to adjacent subgroups (ablation) | CATANet IASA · Manga109 ×4 | PSNR: 31.26 -> 31.31 | Helped |
| add IASA and IRCA outputs instead of concatenating them (ablation) | CATANet · Urban100 ×4 | PSNR: 26.82 -> 26.87 | Helped |
| divide token groups into fixed-size subgroups (ablation) | CATANet-L · inference | inference time: 188ms -> 86ms | Helped |
Surprising findings
- Token centers are updated only during training, so clustering adds no iterative cost at inference.
Limitations
- Inter-group cross-attention uses token centers only during training-derived priors, so the global prior depends on the training distribution.
- Gains over the best baseline are modest on some datasets (e.g., B100 at x4 is equal to SPIN's 27.75 vs 27.70 range).
Experimental setup
Datasets: DIV2K DF2K
Benchmarks: Set5 Set14 B100 Urban100 Manga109
Metrics: PSNR SSIM Params Multi-Adds running time
Baselines: SPIN SwinIR-light ATD-L ELAN-light IMDN RFDN OSFFNet SRFormer-light SeemoRe-T DITN-Tiny
Builds on: SPIN Routing Transformer NLSA SwinIR HPINet
Closest papers by meaning
- Efficient Attention-Sharing Information Distillation Transformer for Lightweight Single Image Super-Resolution (AAAI 2025 · similarity 0.87)
- Transcending the Limit of Local Window: Advanced Super-Resolution Transformer with Adaptive Token Dictionary (CVPR 2024 · similarity 0.87)
- AdaFormer: Efficient Transformer with Adaptive Token Sparsification for Image Super-resolution (AAAI 2024 · similarity 0.86)
- Lightweight Image Super-Resolution via Flexible Meta Pruning (ICML 2024 · similarity 0.85)
- Recursive Generalization Transformer for Image Super-Resolution (ICLR 2024 · similarity 0.84)
- CAMixerSR: Only Details Need More "Attention" (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.