Research map / Architectures and efficiency / Attention and transformers
EfficientTrain++: Generalized Curriculum Learning for Efficient Visual Backbone Training
Yulin Wang, Yang Yue, Rui Lu, Yizeng Han, Shiji Song, Gao Ming Huang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use EfficientTrain++ curriculum instead of standard training | DeiT-Tiny · ImageNet-1K | top-1 accuracy: 72.5% -> 73.7% | Improved |
| use EfficientTrain++ curriculum instead of standard training | CAFormer-M36 · ImageNet-1K | top-1 accuracy: 84.9% -> 85.0% | No clear effect |
| use EfficientTrain++ instead of EfficientTrain | DeiT-Small · ImageNet-1K | top-1 accuracy: 80.4% -> 81.0% | Improved |
| use computational-constrained sequential search instead of greedy search | EfficientTrain++ · curriculum solving cost | GPU-hours: 1088.1 -> 425.8 | Improved |
| spend the same 300-epoch training cost with EfficientTrain++ | DeiT-Tiny · ImageNet-1K | top-1 accuracy: 72.5% -> 74.4% | Improved |
| apply EfficientTrain++ with a 100-epoch budget | DeiT-Small · ImageNet-1K | top-1 accuracy: 76.7% -> 79.8% | Improved |
| low-pass filter training data in first 75 epochs (r=40) (ablation) | DeiT-Small · ImageNet-1K | top-1 accuracy: 80.3% -> 80.3% | No clear effect |
| low-pass filter training data for 225 epochs (r=40) (ablation) | DeiT-Small · ImageNet-1K | top-1 accuracy: 80.3% -> 79.4% | Hurt |
| filter training data for all 300 epochs (r=40) (ablation) | DeiT-Small · ImageNet-1K | top-1 accuracy: 80.3% -> 78.3% | Hurt |
| use weaker RandAug (m=5) in first 150 epochs (ablation) | DeiT-Small · ImageNet-1K | top-1 accuracy: 80.3% -> 80.7% | Improved |
| add low-frequency cropping to weaker RandAug curriculum (m=9) (ablation) | DeiT-Small · ImageNet-1K | top-1 accuracy: 80.3% -> 79.8% | Hurt |
| use down-sampling instead of low-frequency cropping (ablation) | DeiT-Small · ImageNet-1K | top-1 accuracy | Hurt |
Closest papers by meaning
- From Prototypes to General Distributions: An Efficient Curriculum for Masked Image Modeling (CVPR 2025 · similarity 0.88)
- Beyond Random Augmentations: Pretraining with Hard Views (ICLR 2025 · similarity 0.87)
- Synergy and Diversity in CLIP: Enhancing Performance Through Adaptive Backbone Ensembling (ICLR 2025 · similarity 0.87)
- ScaleKD: Strong Vision Transformers Could Be Excellent Teachers (NeurIPS 2024 · similarity 0.86)
- FastDINOv2: Frequency Based Curriculum Learning Improves Robustness and Training Speed (NeurIPS 2025 · similarity 0.86)
- Data-efficient Large Vision Models through Sequential Autoregression (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
