Research map / Label-efficient and robust learning / Domain adaptation and generalization
Ensemble of Averages: Improving Model Selection and Boosting Performance in Domain Generalization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add simple moving average of weights (SMA, t0=100, every iteration) | ERM ResNet-50 · DomainBed 5-dataset average | accuracy: 64.0 -> 66.5 | Helped |
| ensemble moving-average models (EoA) instead of unaveraged ensemble (ablation) | Ensemble ResNet-50 · DomainBed 5-dataset average | accuracy: 66.8 -> 68.0 | Helped |
| use EoA instead of ERM | ERM ResNet-50 · DomainBed 5-dataset average | accuracy: 64.0 -> 68.0 | Helped |
| use SMA vs SWAD (no hyperparameters) | SWAD · DomainBed 5-dataset average | accuracy: 66.9 -> 66.5 | No clear effect |
| use SMA instead of unaveraged ensemble | Ensemble ResNet-50 · DomainBed 5-dataset average | accuracy: 66.8 -> 66.5 | No clear effect |
| larger pre-training data and model size (ResNeXt then RegNetY) with EoA | EoA · DomainBed 5-dataset average | gain over ERM: 68.0 -> 76.6 (EoA across ResNet-50 to RegNetY) | Helped |
| add model averaging | ERM ResNet-50 · PACS in-domain split | accuracy: 94.39 -> 96.77 | Helped |
| add model averaging | ERM ResNet-50 · OfficeHome in-domain split | accuracy: 77.09 -> 83.56 | Helped |
| add model averaging for rank correlation between validation and test accuracy (ablation) | individual models · TerraIncognita L100 | Spearman correlation: 0.21 -> 0.90 | Helped |
| average models in ensembles for rank correlation (ablation) | ensemble · TerraIncognita | Spearman correlation: 0.59 -> 0.38 (L43), 0.48 -> 1 (L100) | Mixed |
| start averaging close to initialization rather than later (ablation) | SMA · DomainBed (pretrained init) | out-domain accuracy | Helped |
| reduce averaging frequency (sample at larger intervals) (ablation) | SMA · DomainBed | accuracy | No clear effect |
Closest papers by meaning
- Understanding Domain Generalization: A Noise Robustness Perspective (ICLR 2024 · similarity 0.87)
- Sharpness-Aware Model-Agnostic Long-Tailed Domain Generalization (AAAI 2024 · similarity 0.87)
- Set Valued Predictions For Robust Domain Generalization (ICML 2025 · similarity 0.87)
- PEER Pressure: Model-to-Model Regularization for Single Source Domain Generalization (CVPR 2025 · similarity 0.87)
- When, Where and Why to Average Weights? (ICML 2025 · similarity 0.86)
- Lost Domain Generalization Is a Natural Consequence of Lack of Training Domains (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.