AI Paper Atlas

Research map / Architectures and efficiency / Attention and transformers

EfficientTrain++: Generalized Curriculum Learning for Efficient Visual Backbone Training

Yulin Wang, Yang Yue, Rui Lu, Yizeng Han, Shiji Song, Gao Ming Huang

TPAMI 2024 · Architectures and efficiency › Attention and transformers · approach: Token merging and masked pretraining · 26 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use EfficientTrain++ curriculum instead of standard trainingDeiT-Tiny · ImageNet-1Ktop-1 accuracy: 72.5% -> 73.7%Improved
use EfficientTrain++ curriculum instead of standard trainingCAFormer-M36 · ImageNet-1Ktop-1 accuracy: 84.9% -> 85.0%No clear effect
use EfficientTrain++ instead of EfficientTrainDeiT-Small · ImageNet-1Ktop-1 accuracy: 80.4% -> 81.0%Improved
use computational-constrained sequential search instead of greedy searchEfficientTrain++ · curriculum solving costGPU-hours: 1088.1 -> 425.8Improved
spend the same 300-epoch training cost with EfficientTrain++DeiT-Tiny · ImageNet-1Ktop-1 accuracy: 72.5% -> 74.4%Improved
apply EfficientTrain++ with a 100-epoch budgetDeiT-Small · ImageNet-1Ktop-1 accuracy: 76.7% -> 79.8%Improved
low-pass filter training data in first 75 epochs (r=40) (ablation)DeiT-Small · ImageNet-1Ktop-1 accuracy: 80.3% -> 80.3%No clear effect
low-pass filter training data for 225 epochs (r=40) (ablation)DeiT-Small · ImageNet-1Ktop-1 accuracy: 80.3% -> 79.4%Hurt
filter training data for all 300 epochs (r=40) (ablation)DeiT-Small · ImageNet-1Ktop-1 accuracy: 80.3% -> 78.3%Hurt
use weaker RandAug (m=5) in first 150 epochs (ablation)DeiT-Small · ImageNet-1Ktop-1 accuracy: 80.3% -> 80.7%Improved
add low-frequency cropping to weaker RandAug curriculum (m=9) (ablation)DeiT-Small · ImageNet-1Ktop-1 accuracy: 80.3% -> 79.8%Hurt
use down-sampling instead of low-frequency cropping (ablation)DeiT-Small · ImageNet-1Ktop-1 accuracyHurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy