Research map / Video / Video-language understanding
Prune Spatio-temporal Tokens by Semantic-aware Temporal Accumulation
Shuangrui Ding, Peisen Zhao, Xiaopeng Zhang, Rui Qian, Hongkai Xiong, Qi Tian
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply STA token pruning to off-the-shelf ViT-H without retraining | ViT-H · Kinetics-400 | Top-1 accuracy / GFLOPs: GFLOPs 1192 -> 611; accuracy 86.3 -> 86.1 | Improved |
| apply STA to VideoSwin-B with the same backbone vs STTS | VideoSwin-B · Kinetics-400 | Top-1 accuracy: 81.9 -> 82.3 (149 vs 253 GFLOPs) | Improved |
| apply STA to VideoSwin-B vs STTS | VideoSwin-B · Something-Something V2 | Top-1 accuracy: 68.7 -> 69.2 (188 vs 237 GFLOPs) | Improved |
| increase first-stage drop number r1 for small models (ViT-S to 64) (ablation) | ViT-S · Kinetics-400 | Top-1 accuracy: 78.8 -> 78.1 | Hurt |
| combine temporal accumulation score with semantic score instead of either alone (ablation) | STA on ViT-L · Kinetics-400 | Top-1 accuracy: 84.69 / 84.87 -> 84.96 | Improved |
| use decreasing drop schedule instead of constant (ablation) | STA on ViT-L · Kinetics-400 | Top-1 accuracy: 84.68 -> 84.96 | Improved |
| use increasing drop schedule instead of decreasing (ablation) | STA on ViT-L · Kinetics-400 | Top-1 accuracy: 84.96 -> 77.68 | Hurt |
| alternate accumulation order (forward-backward-forward) instead of the same direction (ablation) | STA on ViT-L · Kinetics-400 | Top-1 accuracy: 84.26 -> 84.96 | Improved |
| choose first-frame pruning method (random, grid, ToMe) (ablation) | STA on ViT-L · Kinetics-400 | Top-1 accuracy: 84.78 -> 84.96 | No clear effect |
| use key projection f_K for token affinity instead of f_Q, f_V or f_FFN (ablation) | STA on ViT-L · Kinetics-400 | Top-1 accuracy: 84.83 -> 84.96 | No clear effect |
| train ViT-B with STA to halve training time | ViT-B · Kinetics-400 | Top-1 accuracy: 81.2 -> 80.6 | Hurt |
Closest papers by meaning
- Exploring Token Pruning in Vision State Space Models (NeurIPS 2024 · similarity 0.87)
- Token Cropr: Faster ViTs for Quite a Few Tasks (CVPR 2025 · similarity 0.86)
- VLM-Pruner: Buffering for Spatial Sparsity in an Efficient VLM Centrifugal Token Pruning Paradigm (CVPR 2026 · similarity 0.86)
- ImagePiece: Content-aware Re-tokenization for Efficient Image Recognition (AAAI 2025 · similarity 0.86)
- Efficient Time Series Processing for Transformers and State-Space Models through Token Merging (ICML 2025 · similarity 0.85)
- V-CAST: Video Curvature-Aware Spatio-Temporal Pruning for Efficient Video Large Language Models (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
