Research map / Architectures and efficiency / Attention and transformers
Removing Rows and Columns of Tokens in Vision Transformer enables Faster Dense Prediction without Retraining
Diwei Su, cheng fei, Jianxu Luo
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply Token Adapter to Segmenter at best-accuracy setting | Segmenter ViT-L/16 · ADE20K | mIoU: 51.82 -> 52.02 with FPS 10.68 -> 15.09 | Improved |
| replace clustering (Expediting) with matching (Token Adapter) at max speedup | Segmenter ViT-L/16 · ADE20K | mIoU: 43.86 -> 45.73 | Improved |
| replace ToMe with Token Adapter at max speedup | Segmenter ViT-L/16 · Cityscapes | FPS: 13.12 -> 19.38 | Improved |
| apply Token Adapter instead of Expediting at early MaskDINO stages | MaskDINO Swin-L · COCO panoptic | PQ: 30.7 -> 55.91 (stage 1) | Improved |
| apply Token Adapter at stage 4 of MaskDINO | MaskDINO Swin-L · COCO panoptic | PQ: 58.3 -> 58.42 | No clear effect |
| use Token Adapter instead of Token Merging at aggressive r=64 on SWAG | SWAG ViT-L/16 · ImageNet-1k val | accuracy: 67.56 -> 85.34 | Improved |
| select rows/columns by feature importance scores instead of random (ablation) | Token Adapter · ADE20K | mIoU: 51.38 -> 52.02 | Improved |
| use feature scores instead of attention scores for selection (ablation) | Token Adapter · ADE20K | mIoU: 51.13 -> 52.02 | Improved |
| use cosine distance instead of other metrics for matching (ablation) | Token Adapter · ADE20K | mIoU: 51.80 -> 52.02 (dot) | Improved |
| merge by averaging instead of pooling or bilinear resize (ablation) | Token Adapter · ADE20K | mIoU: 48.15 -> 52.02 (max pool) | Improved |
| merge normal tokens instead of just keeping representative and unique tokens (ablation) | Token Adapter · ADE20K | mIoU: 50.67 -> 52.02 | Improved |
| use threshold matching in ejector instead of one-to-one max (ablation) | Token Adapter · ADE20K | mIoU: 51.93 -> 52.02 | Improved |
Closest papers by meaning
- Token Cropr: Faster ViTs for Quite a Few Tasks (CVPR 2025 · similarity 0.90)
- Multi-criteria Token Fusion with One-step-ahead Attention for Efficient Vision Transformers (CVPR 2024 · similarity 0.89)
- DTMFormer: Dynamic Token Merging for Boosting Transformer-Based Medical Image Segmentation (AAAI 2024 · similarity 0.89)
- ImagePiece: Content-aware Re-tokenization for Efficient Image Recognition (AAAI 2025 · similarity 0.88)
- Frequency-Aware Token Reduction for Efficient Vision Transformer (NeurIPS 2025 · similarity 0.87)
- Learning to Merge Tokens via Decoupled Embedding for Efficient Vision Transformers (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
