Research map / Architectures and efficiency / Attention and transformers
Evaluating Robustness of Vision Transformers on Imbalanced Datasets (Student Abstract)
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DeiT-T instead of ResNet-32 on imbalanced CIFAR-10 and CIFAR-100 | CIFAR image classification · CIFAR-10 and CIFAR-100, 10× and 100× | Top-1 accuracy: CIFAR-10: 0.867 -> 0.721; 0.712 -> 0.467; CIFAR-100: 0.567 -> 0.463; 0.389 -> 0.254 | Hurt |
| Use DeiT-S instead of ResNet-50 | ImageNet-LT image classification · ImageNet-LT | Top-1 accuracy: 0.442 -> 0.311 | Hurt |
| Defer class-level loss reweighting until after 80% of training rather than start it immediately | CNN and ViT classifiers · CIFAR-10, CIFAR-100, and ImageNet-LT | Top-1 accuracy: CNN: 0.872 -> 0.877; 0.707 -> 0.757; 0.559 -> 0.578; 0.350 -> 0.415; 0.390 -> 0.476. ViT: 0.720 -> 0.759; 0.538 -> 0.556 | Helped |
| Add class-level loss reweighting from the start of training | DeiT-T and DeiT-S · Imbalanced datasets | Top-1 accuracy: RW versus baseline: CIFAR-10 10× 0.721 -> 0.720; 100× 0.467 -> 0.538; CIFAR-100 10× 0.463 -> 0.404; 100× 0.254 -> 0.202; | Mixed |
| Allow patch embedding to update during DRW rather than freeze it (ablation) | DeiT-T · CIFAR-10 10× and 100× | Top-1 accuracy: 0.756 -> 0.759; 0.555 -> 0.556 | Helped |
| Allow self-attention to update during DRW rather than freeze patch embedding and self-attention (ablation) | DeiT-T · CIFAR-10 10× and 100× | Top-1 accuracy: 0.738 -> 0.759; 0.534 -> 0.556 | Helped |
| Allow all components, including the linear classifier, to update during DRW (ablation) | DeiT-T · CIFAR-10 10× and 100× | Top-1 accuracy: 0.699 -> 0.759; 0.447 -> 0.556 | Helped |
| Allow patch embedding to update during DRW rather than freeze it (ablation) | DeiT-T · CIFAR-100 10× | Top-1 accuracy: 0.493 -> 0.494 | Helped |
| Allow self-attention to update during DRW rather than freeze patch embedding and self-attention (ablation) | DeiT-T · CIFAR-100 10× | Top-1 accuracy: 0.479 -> 0.494 | Helped |
Closest papers by meaning
- Vision Transformers Are Robust Learners (AAAI 2022 · similarity 0.88)
- Learning Imbalanced Data With Vision Transformers (CVPR 2023 · similarity 0.88)
- On the Learning Property of Logistic and Softmax Losses for Deep Neural Networks (AAAI 2020 · similarity 0.88)
- The Principle of Diversity: Training Stronger Vision Transformers Calls for Reducing All Levels of Redundancy (CVPR 2022 · similarity 0.88)
- A Theoretical Analysis of Self-Supervised Learning for Vision Transformers (ICLR 2025 · similarity 0.87)
- Gradient Reweighting: Towards Imbalanced Class-Incremental Learning (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.