Research map / Architectures and efficiency / Attention and transformers
Learning Imbalanced Data With Vision Transformers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the balanced logit bias to BCE (ablation) | ViT-Base with BCE · ImageNet-LT | Top-1 accuracy: 52.4 -> 60.9 | Helped |
| Add the balanced logit bias to BCE (ablation) | ViT-Base with BCE · ImageNet-LT | ECE: 5.6 -> 2.4 | Helped |
| Use balanced BCE rather than balanced CE (ablation) | ViT-Base · ImageNet-LT | Top-1 accuracy: 60.1 -> 60.9 | Helped |
| Use balanced BCE rather than CE (ablation) | ViT-Base · ImageNet-LT | Top-1 accuracy: 54.5 -> 60.9 | Helped |
| Use MGP on long-tailed rather than balanced pretraining data (ablation) | ViT-B with Bal-BCE transfer · iNaturalist 2018 | Top-1 accuracy: 59.0 -> 60.7 | Helped |
| Use 384 rather than 224 resolution for BFT | LiVT ViT-Base · ImageNet-LT | Top-1 accuracy: 60.9 -> 63.8 | Helped |
| Use 384 rather than 224 resolution for BFT | LiVT ViT-Base · iNaturalist 2018 | Top-1 accuracy: 76.1 -> 81.0 | Helped |
| Use 384 rather than 224 resolution for BFT | LiVT ViT-Base · Places-LT | Top-1 accuracy: 40.8 -> 42.6 | Helped |
| Increase the bias scale τ (ablation) | LiVT on CIFAR-100-LT · CIFAR-100-LT | Few-shot accuracy | Helped |
| Increase the bias scale τ beyond its overall-optimal setting (ablation) | LiVT on CIFAR-100-LT · CIFAR-100-LT | overall accuracy | Hurt |
| Pretrain with MGP rather than label-supervised CE (ablation) | ViT-B transfer model · iNaturalist 2018 | Top-1 accuracy: 55.9 -> 59.0 | Helped |
| Use balanced BCE rather than CE during transfer (ablation) | ViT-B · iNaturalist 2018 | Top-1 accuracy: 55.9 -> 59.0 | Helped |
Closest papers by meaning
- DeiT-LT: Distillation Strikes Back for Vision Transformer Training on Long-Tailed Datasets (CVPR 2024 · similarity 0.89)
- Long-tailed Recognition with Model Rebalancing (NeurIPS 2025 · similarity 0.88)
- DiffuLT: Diffusion for Long-tail Recognition Without External Knowledge (NeurIPS 2024 · similarity 0.87)
- Feature Fusion from Head to Tail for Long-Tailed Visual Recognition (AAAI 2024 · similarity 0.86)
- Inverse Weight-Balancing for Deep Long-Tailed Learning (AAAI 2024 · similarity 0.86)
- A Theoretical Analysis of Self-Supervised Learning for Vision Transformers (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.