Research map / Label-efficient and robust learning / Recognition and long-tail
No One Representation to Rule Them All: Overlapping Features of Training Methods
Raphael Gontijo-Lopes, Yann Dauphin, Ekin Dogus Cubuk
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Combine models trained with increasingly different methodologies | ResNet-50 ensembles · ImageNet | ensemble accuracy: 76.5% -> 83.4% | Improved |
| Add a lower-accuracy model trained with a different objective and dataset | ResNet-50 · ImageNet | top-1 accuracy: 76.2% -> 77.7% | Improved |
| Greedily add only lower-accuracy models | BiT-1k ensemble · ImageNet | ensemble accuracy: 82.85% -> 86.66% | Improved |
| Train models with more divergent methodologies rather than only changing initialization | Model pairs relative to ResNet-50 · ImageNet | error inconsistency: 0.0850 -> 0.2295 | Improved |
| Combine embeddings from differently trained models rather than use either embedding alone | ImageNet linear evaluation · ImageNet | LBFGS accuracy | Improved |
| Use more divergent training methodologies to produce representations, then select diverse dimensions | ImageNet feature representations · ImageNet | features needed for 76% accuracy: 2304 -> 512 | Improved |
| Combine features from differently trained models | Stacked generalization · Pascal VOC | 11-point mAP: 91.4% | Improved |
| Use class-aware interpolation for ResNet-50 and CLIP-S logits | ResNet-50 + CLIP-S ensemble · ImageNet | accuracy: 80.424% -> 80.452% | Improved |
Closest papers by meaning
- Understanding Overadaptation in Supervised Fine-Tuning: The Role of Ensemble Methods (ICML 2025 · similarity 0.87)
- Theoretical Limitations of Ensembles in the Age of Overparameterization (ICML 2025 · similarity 0.87)
- Gradient-Weight Alignment as a Train-Time Proxy for Generalization in Classification Tasks (NeurIPS 2025 · similarity 0.87)
- Feature Averaging: An Implicit Bias of Gradient Descent Leading to Non-Robustness in Neural Networks (ICLR 2025 · similarity 0.86)
- Objective drives the consistency of representational similarity across datasets (ICML 2025 · similarity 0.86)
- Evaluating Model Bias Requires Characterizing its Mistakes (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
