Research map / Label-efficient and robust learning / Recognition and long-tail
On the Trade-off of Intra-/Inter-class Diversity for Supervised Pre-training
Jieyu Zhang, Bohan Wang, Zhengyu Hu, Pang Wei Koh, Alexander Ratner
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| estimate class count via Extrapolation instead of Standard K=1000 | ImageNet subset pre-training, N=50K · CIFAR10 | test error rate: 29.19 -> 26.27 | Improved |
| Extrapolation instead of Standard K=1000 | pre-training N=50K · StanfordDogs | test error rate: 64.96 -> 60.15 | Improved |
| Extrapolation instead of Standard K=1000 | pre-training N=50K · Stanford40 | test error rate: 62.45 -> 57.25 | Improved |
| Extrapolation instead of Standard K=1000 | pre-training N=100K · CIFAR10 | test error rate: 25.04 -> 23.12 | Improved |
| Extrapolation instead of Standard K=1000 | pre-training N=100K · StanfordDogs | test error rate: 54.70 -> 53.17 | Improved |
| Extrapolation (cheap, N=5000 estimate) instead of Grid Search | pre-training N=50K · MIT67 | test error rate: 54.10 -> 53.01 | No clear effect |
| Extrapolation instead of Grid Search | pre-training N=100K · StanfordDogs | test error rate: 51.98 -> 53.17 | Hurt |
| fewer classes than all 1000 ImageNet classes at fixed N | supervised pre-training ResNet-18 · six downstream tasks | test error rate: 29.19 -> 26.24 | Improved |
| fit theoretical error function to choose K (Fitting) vs Standard | pre-training N=100K · CIFAR10 | test error rate: 25.04 -> 22.67 | Improved |
| predicted vs actual error from fitted function | Fitting equation · CIFAR10 N=50K | test error rate: 26.13 vs 26.25 | No clear effect |
| predicted vs actual error from fitted function | Fitting equation · StanfordDogs N=100K | test error rate: 51.69 vs 52.24 | Mixed |
| vary class count by clustering fixed samples (data process 2) instead of sampling new data | supervised pre-training · six downstream tasks | test error rate | Improved |
Closest papers by meaning
- Quick-Tune: Quickly Learning Which Pretrained Model to Finetune and How (ICLR 2024 · similarity 0.86)
- Balanced Data, Imbalanced Spectra: Unveiling Class Disparities with Spectral Imbalance (ICML 2024 · similarity 0.86)
- Understanding and Mitigating the Label Noise in Pre-training on Downstream Tasks (ICLR 2024 · similarity 0.86)
- Provable Benefits of Unsupervised Pre-training and Transfer Learning via Single-Index Models (ICML 2025 · similarity 0.85)
- Class-Attribute Priors: Adapting Optimization to Heterogeneity and Fairness Objective (AAAI 2024 · similarity 0.84)
- Diversified Batch Selection for Training Acceleration (ICML 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
