Research map / Label-efficient and robust learning / Self-supervised and contrastive
EfficientTrain: Exploring Generalized Curriculum Learning for Training Visual Backbones
Yulin Wang, Yang Yue, Rui Lu, Tianjiao Liu, Zhao Zhong, Shiji Song, Gao Huang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train with EfficientTrain (low-frequency cropping curriculum plus linear RandAug) instead of standard training | various backbones · ImageNet-1K | top-1 accuracy and training cost: 83.4% -> 83.6% (Swin-Base) at 1.50x speedup | Improved |
| add linear RandAug on top of low-frequency cropping (ablation) | EfficientTrain DeiT-Tiny · ImageNet-1K | top-1 accuracy (%): 72.4 -> 73.3 | Improved |
| low-frequency cropping alone (ablation) | baseline training · ImageNet-1K | top-1 accuracy (%): 80.3 -> 80.0 (DeiT-S) at 1.5x speedup | Mixed |
| replace low-frequency cropping with image down-sampling (ablation) | EfficientTrain · ImageNet-1K | top-1 accuracy (%): 81.4 -> 81.0 (Swin-T) | Hurt |
| use searched schedule of B instead of linear increasing (ablation) | EfficientTrain · ImageNet-1K | top-1 accuracy (%): 79.9 -> 80.4 (DeiT-S) | Improved |
| shorter training schedule (100 epochs) with EfficientTrain | DeiT-Small · ImageNet-1K | top-1 accuracy (%): 75.5 -> 76.4 | Improved |
| use EfficientTrain with matched wall-time (450 epochs vs 300) | DeiT-Tiny · ImageNet-1K | top-1 accuracy (%): 72.5 -> 74.3 | Improved |
| increase final input size from 224 to 512 | EfficientTrain Swin-Base · ImageNet-1K | training speedup: 1.50x -> 3.37x | Improved |
| pre-train MAE with EfficientTrain | MAE ViT-B and ViT-L · ImageNet-1K fine-tuning | top-1 accuracy (%): ViT-B 83.5 -> 83.6; ViT-L 85.9 -> 85.8 | No clear effect |
| add FixRes on top of EfficientTrain | CSWin-Small · ImageNet-1K | top-1 accuracy (%): 83.6 -> 83.8 | Improved |
| pre-train detection backbones with EfficientTrain | Swin backbones on COCO · COCO Cascade Mask-RCNN | AP box: Swin-T 48.1 -> 48.2; Swin-S 50.0 -> 50.6 | No clear effect |
Closest papers by meaning
- EfficientTrain++: Generalized Curriculum Learning for Efficient Visual Backbone Training (TPAMI 2024 · similarity 0.96)
- From Prototypes to General Distributions: An Efficient Curriculum for Masked Image Modeling (CVPR 2025 · similarity 0.86)
- Beyond Random Augmentations: Pretraining with Hard Views (ICLR 2025 · similarity 0.86)
- ScaleKD: Strong Vision Transformers Could Be Excellent Teachers (NeurIPS 2024 · similarity 0.86)
- Time- Memory- and Parameter-Efficient Visual Adaptation (CVPR 2024 · similarity 0.85)
- On the Surprising Effectiveness of Attention Transfer for Vision Transformers (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
