Research map / Detection and segmentation / CNN features
Content-Aware Token Sharing for Efficient Semantic Segmentation With Vision Transformers
Chenyang Lu, Daan de Geus, Gijs Dubbelman
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Share tokens for predicted semantically uniform neighboring patches instead of processing every patch separately | Segmenter with ViT-S/16 · ADE20K val | mIoU: 45.0 -> 45.1 | No clear effect |
| Share tokens for predicted semantically uniform neighboring patches instead of processing every patch separately | Segmenter with ViT-S/16 · ADE20K val | Im/sec: 122 -> 162 | Improved |
| Select shared patches with CTS rather than randomly (ablation) | Segmenter with ViT-S/16 · ADE20K val | mIoU: 42.9 -> 45.1 | Improved |
| Use a larger PolicyNet (ablation) | CTS with Segmenter, ViT-S/16 · ADE20K val | mIoU: 45.1 -> 45.3 | Improved |
| Use a smaller custom PolicyNet (ablation) | CTS with Segmenter, ViT-S/16 · ADE20K val | mIoU: 45.1 -> 44.0 | Hurt |
| Generate sharing policy from segmentation predictions rather than the class-agnostic PolicyNet (ablation) | CTS with Segmenter, ViT-S/16 · ADE20K val | mIoU: 45.1 -> 44.3; 45.1 -> 44.8 | Hurt |
| Use a ground-truth segmentation policy instead of predicted PolicyNet policy (ablation) | CTS with Segmenter, ViT-S/16 · ADE20K val | mIoU: 45.1 -> 47.2 | Improved |
| Choose shared patches randomly instead of using a content-aware policy (ablation) | CTS with Segmenter, ViT-S/16 · ADE20K val | mIoU: 45.0 -> 42.9 | Hurt |
| Apply CTS with 30% token reduction | Segmenter with ViT-B/16 · ADE20K val | mIoU: 48.5 -> 48.7 | No clear effect |
| Apply CTS with 30% token reduction | BEiT-L + UPerNet · ADE20K val | Im/sec: 3.3 -> 5.4 | Improved |
| Apply CTS with 30% token reduction | BEiT-L + UPerNet · ADE20K val | mIoU: 56.8 -> 56.6 | No clear effect |
| Apply CTS with 30% token reduction | Segmenter with ViT-S/16 · Pascal Context validation | mIoU: 53.0 -> 52.9 | No clear effect |
Closest papers by meaning
- Learning to Merge Tokens via Decoupled Embedding for Efficient Vision Transformers (NeurIPS 2024 · similarity 0.88)
- ImagePiece: Content-aware Re-tokenization for Efficient Image Recognition (AAAI 2025 · similarity 0.88)
- ALGM: Adaptive Local-then-Global Token Merging for Efficient Semantic Segmentation with Plain Vision Transformers (CVPR 2024 · similarity 0.87)
- Pixels to Tokens: Token Space Efficient Active Learning for Low-Budget Semantic Segmentation (NeurIPS 2026 · similarity 0.87)
- Token Cropr: Faster ViTs for Quite a Few Tasks (CVPR 2025 · similarity 0.87)
- Class Tokens Infusion for Weakly Supervised Semantic Segmentation (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
