Research map / Architectures and efficiency / Attention and transformers
SAViT: Structure-Aware Vision Transformer Pruning via Collaborative Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add cross-component interactions (Hessian-based) to the pruning objective (ablation) | SAViT DeiT-B 70% FLOPs reduction · ImageNet-1k | Top-1 accuracy: 79.68 -> 80.78 | Helped |
| add cross-component interactions to the pruning objective (ablation) | SAViT Faster R-CNN Swin-T 70% FLOPs reduction · COCO | mAP: 44.5 -> 45.2 | Helped |
| prune with SAViT instead of l1-norm at ~50% FLOPs | Swin-B · ImageNet-1k | Top-1 accuracy: 80.95 -> 82.62 | Helped |
| prune with SAViT instead of l1-norm at 70% FLOPs | Faster R-CNN Swin-T · COCO | mAP: 42.1 -> 45.2 | Helped |
| collaborative pruning of all components to 50% FLOPs vs unpruned DeiT-B | DeiT-B · ImageNet-1k | Top-1 accuracy: 81.84 -> 82.54 | Helped |
| prune DeiT-B to 70% FLOPs reduction with SAViT | DeiT-B · ImageNet-1k | Top-1 accuracy: 81.84 -> 81.66 | Hurt |
| fine-tune pruned model with knowledge distillation (SAViT-T vs SAViT-S/NVP) | SAViT-T · ImageNet-1k | Top-1 accuracy: 74.50 -> 76.95 | Helped |
| prune Swin-B by 50% FLOPs with SAViT | Swin-B · ImageNet-1k | Top-1 accuracy: 83.51 -> 82.62 | Hurt |
Closest papers by meaning
- Global Vision Transformer Pruning With Hessian-Aware Saliency (CVPR 2023 · similarity 0.92)
- Rethinking Depth Pruning for Vision Transformers: A Heterogeneity-Aware Perspective (ICML 2026 · similarity 0.91)
- Collaborative Multi-Mode Pruning for Vision-Language Models (CVPR 2026 · similarity 0.89)
- Zero-TPrune: Zero-Shot Token Pruning through Leveraging of the Attention Graph in Pre-Trained Transformers (CVPR 2024 · similarity 0.89)
- MDP: Multidimensional Vision Model Pruning with Latency Constraint (CVPR 2025 · similarity 0.88)
- Data-independent Module-aware Pruning for Hierarchical Vision Transformers (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.