Research map / Architectures and efficiency / Attention and transformers
PSLT: A Light-Weight Vision Transformer With Ladder Self-Attention and Progressive Shift
Gaojie Wu, Wei‐Shi Zheng, Yutong Lu, Qi Chuan Tian
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add shift operation in each branch (ablation) | PSLT · ImageNet-1k val (ablation, from scratch) | top-1 accuracy: 79.3 -> 79.9 | Improved |
| add progressive feature delivery between branches (ablation) | PSLT · ImageNet-1k val (ablation, from scratch) | top-1 accuracy: 79.2 -> 79.9 | Improved |
| add pixel-adaptive fusion module instead of FC fusion (ablation) | PSLT · ImageNet-1k val (ablation, from scratch) | top-1 accuracy: 78.8 -> 79.9 | Improved |
| add pixel-adaptive fusion module instead of concatenation (ablation) | PSLT · ImageNet-1k val (ablation, from scratch) | top-1 accuracy: 77.3 -> 79.9 | Improved |
| use spatial-and-channel adaptive weights instead of channel-only SE (ablation) | PSLT · ImageNet-1k val (ablation, from scratch) | top-1 accuracy: 79.0 -> 79.9 | Improved |
| use Light FFN instead of original FFN (ablation) | PSLT · ImageNet-1k val (ablation, from scratch) | top-1 accuracy: 79.7 -> 79.9 | No clear effect |
| increase branches from 3 to 6 (ablation) | PSLT · ImageNet-1k val (ablation, from scratch) | top-1 accuracy: 79.9 -> 77.9 | Hurt |
| decrease branches from 3 to 2 (ablation) | PSLT · ImageNet-1k val (ablation, from scratch) | top-1 accuracy: 79.9 -> 80.2 | Improved |
| add pixel-adaptive fusion on top of LSA and Light FFN (ablation) | PSLT · ImageNet-V2 | top-1 accuracy: 64.6 -> 68.6 | Improved |
| add ladder self-attention and Light FFN to replace ViT-B (ablation) | PSLT · ImageNet-1k val (from scratch) | top-1 accuracy: 77.9 -> 77.3 | No clear effect |
| use PSLT with 9.0M params instead of PCB (+RPP) for re-id | PSLT · Market-1501 | mAP: 81.6 -> 86.2 | Improved |
| compare PSLT against EfficientNet backbone for detection | PSLT · COCO val2017 RetinaNet 1x | AP: 40.5 -> 41.2 | Improved |
Closest papers by meaning
- FLatten Transformer: Vision Transformer using Focused Linear Attention (ICCV 2023 · similarity 0.88)
- A Survey on Efficient Vision Transformers: Algorithms, Techniques, and Performance Benchmarking (TPAMI 2024 · similarity 0.88)
- You Only Need Less Attention at Each Stage in Vision Transformers (CVPR 2024 · similarity 0.87)
- Breaking the Low-Rank Dilemma of Linear Attention (CVPR 2025 · similarity 0.86)
- PADRe: A Unifying Polynomial Attention Drop-in Replacement for Efficient Vision Transformer (ICLR 2025 · similarity 0.86)
- Skip-Attention: Improving Vision Transformers by Paying Less Attention (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
