Research map / Architectures and efficiency / Attention and transformers
TokenMixup: Efficient Attention-guided Token-level Data Augmentation for Transformers
Hyeong Kyu Choi, Joonmyung Choi, Hyunwoo J. Kim
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use attention-based rather than gradient-based saliency detection | HTM on CCT · CIFAR-100 | accuracy: 83.28 -> 83.56 | Improved |
| Use attention-based rather than gradient-based saliency detection | HTM on CCT · CIFAR-100 saliency detection | Avg. Latency (ms): 236 -> 16 | Improved |
| Add sample difficulty assessment with ScoreNet (ablation) | HTM on CCT · CIFAR-100 | accuracy: 83.56 -> 82.85 | Improved |
| Add Hungarian optimal assignment for mixup pairs (ablation) | HTM on CCT · CIFAR-100 | accuracy: 83.56 -> 82.61 | Improved |
| Use saliency-guided token-level mixup instead of random token replacement (ablation) | HTM on CCT · CIFAR-100 | accuracy: 83.56 -> 83.05 | Improved |
| Add saliency-based label reassignment (ablation) | HTM on CCT · CIFAR-100 | accuracy: 83.56 -> 82.62 | Improved |
| Add Horizontal TokenMixup | CCT-7/3x1 · CIFAR-100 with Input Mixup and Cutmix | accuracy: 82.87 -> 83.56 | Improved |
| Add Vertical TokenMixup | CCT-7/3x1 · CIFAR-100 with Input Mixup and Cutmix | accuracy: 82.87 -> 83.54 | Improved |
| Add both Horizontal and Vertical TokenMixup | CCT-7/3x1 · CIFAR-100 with Input Mixup and Cutmix | accuracy: 82.87 -> 83.57 | Improved |
| Use Manifold Mixup instead of token mixup | CCT-7/3x1 · CIFAR-100 with Input Mixup and Cutmix | accuracy: 82.87 -> 80.08 | Hurt |
Closest papers by meaning
- TokenMix: Rethinking Image Mixing for Data Augmentation in Vision Transformers (ECCV 2022 · similarity 0.91)
- MixPro: Data Augmentation with MaskMix and Progressive Attention Labeling for Vision Transformer (ICLR 2023 · similarity 0.87)
- Token Selection is a Simple Booster for Vision Transformers (TPAMI 2022 · similarity 0.87)
- Learning to Merge Tokens via Decoupled Embedding for Efficient Vision Transformers (NeurIPS 2024 · similarity 0.85)
- Video Token Merging for Long Video Understanding (NeurIPS 2024 · similarity 0.85)
- ImagePiece: Content-aware Re-tokenization for Efficient Image Recognition (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
