Research map / Architectures and efficiency / RNNs and normalization
Diverse Weight Averaging for Out-of-Distribution Generalization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| average weights from different runs (DiWA uniform M=20) instead of moving average along one run | ERM ResNet50, random init · DomainBed PACS | accuracy: 87.9 -> 88.8 | Helped |
| use linear-probing classifier init instead of random init (ablation) | DiWA† M=60 · DomainBed average | accuracy: 67.1 -> 68.0 | Helped |
| restricted weight selection instead of uniform (ablation) | DiWA random init · DomainBed TerraInc | accuracy: 48.9 -> 50.5 | Helped |
| restricted instead of uniform selection with LP init (ablation) | DiWA LP init · DomainBed average | accuracy: 67.6 -> 67.5 | No clear effect |
| scale M from 20 to 60 by averaging weights across 3 data splits (ablation) | DiWA LP init · DomainBed average | accuracy: 67.6 -> 68.0 | Helped |
| add Mixup and Coral objectives to the averaged runs (ablation) | DiWA uniform · OfficeHome Art | accuracy: 67.3 -> 68.4 | Helped |
| weights from different runs instead of single run (ablation) | WA · OfficeHome Art | accuracy | Helped |
| use extreme hyperparameter ranges or non-shared classifier init (ablation) | DiWA · OfficeHome Art | accuracy | Hurt |
| train each DiWA run with SAM (ablation) | DiWA uniform M=20 · OfficeHome Art | accuracy: 67.3 -> 66.7 | Hurt |
| combine SWAD with SAM | SWAD · DomainBed average | accuracy: 66.9 -> 65.5 | Hurt |
| weight averaging on correlation shift | WA · ColoredMNIST | accuracy | No clear effect |
| weight averaging (DiWA M=20) instead of ensembling of predictions | DiWA random init · DomainBed average | accuracy: 67.1 -> 66.8 | No clear effect |
Closest papers by meaning
- When, Where and Why to Average Weights? (ICML 2025 · similarity 0.86)
- DELT: A Simple Diversity-driven EarlyLate Training for Dataset Distillation (CVPR 2025 · similarity 0.85)
- Diversity-Driven Synthesis: Enhancing Dataset Distillation through Directed Weight Adjustment (NeurIPS 2024 · similarity 0.85)
- Gradient-Weight Alignment as a Train-Time Proxy for Generalization in Classification Tasks (NeurIPS 2025 · similarity 0.85)
- Feature Averaging: An Implicit Bias of Gradient Descent Leading to Non-Robustness in Neural Networks (ICLR 2025 · similarity 0.85)
- DaWin: Training-free Dynamic Weight Interpolation for Robust Adaptation (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.