Research map / Detection and segmentation / CNN features
Token Cropr: Faster ViTs for Quite a Few Tasks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Cropr, a cross-attention token pruner for ViTs whose auxiliary heads learn task-relevant token selection end-to-end and are discarded at inference, combined with Last Layer Fusion to reintroduce pruned tokens for dense tasks.
Problem
Improving ViT inference throughput by token reduction that is fast, keeps high performance, and works across vision tasks including dense prediction (segmentation, detection) where pixel-level outputs conflict with pruning.
Approach
Cropr modules use a streamlined cross-attention scorer with learnable queries, trained via auxiliary task heads with stop-gradient so the encoder is unaffected; at inference the aggregator and heads are removed, leaving a cheap scorer plus Top-K router. Last Layer Fusion concatenates pruned tokens back before the final block so dense predictions can be made.
Key results
- On ImageNet-1k, EVA-02 + Cropr reaches 89.7% top-1 (vs 89.9% unpruned) with 2.1x speedup and ~41% fewer FLOPs.
- On ADE20k, Cropr achieves 56.6% median mIoU across 5 seeds, 0.1 below the no-pruning baseline (56.7%), while being 2.0x faster.
- On COCO with EVA-02-L and Cascade Mask R-CNN, Cropr obtains 63.0 APbox and 54.0 APmask with 1.9x overall speedup, outperforming random, variance and Attn Top-K pruning.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| prune 8 tokens per block with LLF | ViT-L with Cropr · ImageNet-1k | accuracy: 85.8 -> 85.3 | Hurt |
| use Cropr token pruning instead of no pruning | EVA-02-L · ImageNet-1k | accuracy: 89.9 -> 89.7 | Hurt |
| prune 40 tokens after each of the first 22 blocks with LLF | EVA-02-L Segmenter · ADE20k | median mIoU: 56.7% -> 56.6% | Hurt |
| apply five-stage Cropr pruning with LLF | EVA-02-L Cascade Mask R-CNN · COCO val | APbox: 64.2 -> 63.0 | Hurt |
| use 16-head MHA with QKV and head projections and LayerNorm instead of simple single-head cross-attention (ablation) | Cropr · ImageNet-1k | accuracy: 85.3 -> 85.2 | Hurt |
| sample tokens from the cross-attention distribution instead of using Top-K (ablation) | Cropr · ImageNet-1k | accuracy: 85.3 -> 85.1 | Hurt |
| remove the aggregator MLP, LayerNorm, and residual connection (ablation) | Cropr · ImageNet-1k | accuracy: 85.0 -> 85.3 | Hurt |
| allow gradients from Cropr components to flow through the encoder (ablation) | Cropr · ImageNet-1k | accuracy: 85.0 -> 85.3 | Hurt |
| replace LLF with cross-attention fusion without reactivating pruned tokens (ablation) | Cropr · ADE20k | mIoU: 56.6 -> 47.7 | Hurt |
| replace LLF with token concatenation after the last layer (ablation) | Cropr · ADE20k | mIoU: 56.6 -> 51.8 | Hurt |
| replace LLF with cross-attention plus token concatenation (ablation) | Cropr · ADE20k | mIoU: 56.6 -> 51.5 | Hurt |
| replace LLF with MHSA plus token concatenation (ablation) | Cropr · ADE20k | mIoU: 56.6 -> 55.2 | Hurt |
Surprising findings
- Random pruning with Last Layer Fusion achieves decent segmentation performance, and even acts as an effective detector on COCO.
- Stopping gradient flow from auxiliary heads into the encoder improves accuracy (85.3 vs 85.0 on ImageNet ablation).
Limitations
- Speedups and accuracy drops vary with schedule; aggressive pruning (80%) without LLF causes a 1.1-point drop on EVA-02.
- Method requires task-specific auxiliary heads designed per task (classification, segmentation, detection via multi-label proxy).
Experimental setup
Datasets: ImageNet-1k ADE20k COCO Objects365
Benchmarks: ImageNet-1k ADE20k COCO
Metrics: top-1 accuracy median mIoU APbox APmask FLOPs throughput (im/s)
Baselines: EViT ToMe DToP ATS DynamicViT Variance pruning Attn Top-K Random pruning K-Medoids SiT PatchMerger Sinkhorn
Builds on: Perceiver IO Segmenter Cascade Mask R-CNN EVA-02 Masked autoencoder
Closest papers by meaning
- ImagePiece: Content-aware Re-tokenization for Efficient Image Recognition (AAAI 2025 · similarity 0.90)
- Exploring Token Pruning in Vision State Space Models (NeurIPS 2024 · similarity 0.88)
- Balanced Token Pruning: Accelerating Vision Language Models Beyond Local Optimization (NeurIPS 2025 · similarity 0.88)
- When and What to Prune? Stage-Aware Visual Token Pruning for Efficient VLA (NeurIPS 2026 · similarity 0.88)
- VLM-Pruner: Buffering for Spatial Sparsity in an Efficient VLM Centrifugal Token Pruning Paradigm (CVPR 2026 · similarity 0.88)
- TopV: Compatible Token Pruning with Inference Time Optimization for Fast and Low-Memory Multimodal Vision Language Model (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.