Research map / Image generation / Super-resolution
UPS: Unified Projection Sharing for Lightweight Single-Image Super-resolution and Beyond
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes UPS, which decouples per-layer deep feature extraction from similarity modeling by sharing one learnable projection space across all self-attention layers of a lightweight Swin-style SISR network.
Problem
Lightweight transformer-based single-image super-resolution suffers because each layer must jointly optimize deep feature extraction and similarity (projection) modeling under a tight parameter budget, causing co-adaptation and inferior performance.
Approach
Replace per-layer query/key projections with one layer-invariant learnable projection matrix shared across all D-STL layers, computing cosine similarity followed by ReLU in that space, so per-layer updates only handle feature extraction.
Key results
- UPS achieves 39.62 dB on Manga109 (x2) vs 39.12 dB for SwinIR-light with 843K vs 910K parameters, and 0.55 dB gain on Urban100 x2 over SwinIR-light.
- UPS reduces FLOPs by about 33% versus SwinIR-light (163G vs 244G at 1280x720 output) and inference time from 175 ms to 119 ms on RTX 2080Ti.
- On NYU V2 depth SR without depth training, UPS gains 0.54 dB at x4 and 0.73 dB at x16 over SwinIR-light; on degraded Set14 (noise), UPS outperforms by up to 0.36 dB.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace layer-specific projections with one unified projection space | lightweight SISR · Manga109 ×2 | PSNR (dB): 39.29 -> 39.62 | Helped |
| replace layer-specific projections with one unified projection space (ablation) | UPS-S · Set14 ×2 | PSNR (dB): 33.63 -> 34.00 | Helped |
| use 4 projection groups instead of one unified group (ablation) | UPS-S · Set14 ×2 | PSNR (dB): 34.00 -> 33.91 | Hurt |
| use 2 projection groups instead of one unified group (ablation) | UPS-S · Set14 ×2 | PSNR (dB): 34.00 -> 33.97 | Hurt |
| use Cosine similarity with ReLU instead of matrix dot product with SoftMax (ablation) | UPS-S · Urban100 ×2 | PSNR (dB): 33.60 -> 34.00 | Helped |
| increase projection dimension from 8 to 128 (ablation) | UPS-S · Set14 ×2 | PSNR (dB): 33.83 -> 34.00 | Helped |
| increase projection dimension from 128 to 256 (ablation) | UPS-S · Set14 ×2 | PSNR (dB): 34.00 -> 33.96 | Hurt |
| replace ReLU with Softmax activation (ablation) | UPS · Urban100 ×4 | PSNR (dB): 26.83 -> 26.79 | Hurt |
| remove the linear projection for V (ablation) | UPS · Urban100 ×4 | PSNR (dB): 26.80 -> 26.83 | No clear effect |
| remove the linear projection for V (ablation) | UPS · Set14 ×4 | PSNR (dB): 28.91 -> 28.90 | No clear effect |
Surprising findings
- Layer-to-layer CKA similarity of projection spaces in trained SwinIR-light exceeds 0.9 across layers, motivating a shared projection.
- Replacing softmax with ReLU contributes only about 0.04 dB of the 0.36 dB gain; the authors attribute most improvement to the unified projection design.
Limitations
- Larger UPS-based models were not investigated (acknowledged in appendix A.7).
- Real-world and other low-level task extensions are preliminary; no error bars or statistical significance reported.
- Gains over SwinIR-light on some benchmarks (e.g., Set5 x3, Set14) are small, and the paper reports no variance across runs.
Experimental setup
Datasets: DIV2K DF2K NYU V2
Benchmarks: Set5 Set14 BSD100 Urban100 Manga109 RealSRSet NYU V2
Metrics: PSNR SSIM NIQE NRQM PI RMSE inference time FLOPs GPU memory
Baselines: SwinIR-light NGSwin DLGSA-light Omni-SR LatticeNet LAPAR-A IMDN RFDN-L
Builds on: SwinIR ReLUFormer Swin Transformer
Closest papers by meaning
- ResDiff: Combining CNN and Diffusion Model for Image Super-resolution (AAAI 2024 · similarity 0.85)
- Noise-Free Optimization in Early Training Steps for Image Super-resolution (AAAI 2024 · similarity 0.84)
- Edge-SD-SR: Low Latency and Parameter Efficient On-device Super-Resolution with Stable Diffusion via Bidirectional Conditioning (CVPR 2025 · similarity 0.84)
- Efficient Attention-Sharing Information Distillation Transformer for Lightweight Single Image Super-Resolution (AAAI 2025 · similarity 0.83)
- Transformer-Based Selective Super-resolution for Efficient Image Refinement (AAAI 2024 · similarity 0.83)
- Recursive Generalization Transformer for Image Super-Resolution (ICLR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.