Research map / Label-efficient and robust learning / Domain adaptation and generalization
Balanced Direction from Multifarious Choices: Arithmetic Meta-Learning for Domain Generalization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Arith, a first-order meta-learning method for domain generalization that uses arithmetically decreasing inner-loop gradient weights so the outer update approximates the centroid of domain-specific optimal parameters while preserving gradient matching.
Problem
Domain generalization under distribution shift: existing first-order meta-learning (MLDG, Fish) achieves gradient matching across source domains but places balanced parameters without regard to their position relative to each domain's optimum.
Approach
Reformulates the outer update as averaging all intermediate inner-loop models, which in gradient form yields arithmetically decreasing weights (n+1-i) on domain gradients; this approximates the centroid of domain-optimal parameters, and inner-loop SGD without momentum plus Adam in the outer loop keeps domain-wise gradient matching.
Key results
- Arith achieves 65.0% average accuracy across five DomainBed datasets, versus 63.9% for Fish and 64.5% for CORAL.
- On CAMELYON17, Arith reaches 76.6% average accuracy versus 74.8% for Fish and 73.1% for ERM.
- Combining Arith with SWAD raises the average from 65.9 (ERM+SWAD) to 66.7 in the authors' implementation (Arith+SWAD vs. Fish+SWAD 66.5).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use arithmetic meta-learning instead of CORAL | ResNet50 · PACS | accuracy: 86.2 -> 86.5 ± 0.3 | Helped |
| use arithmetic meta-learning instead of CORAL | ResNet50 · VLCS | accuracy: 78.8 -> 79.4 ± 0.3 | Helped |
| use arithmetic meta-learning instead of CORAL | ResNet50 · Office-Home | accuracy: 68.7 -> 69.4 ± 0.1 | Helped |
| use arithmetic meta-learning instead of Fishr | ResNet50 · TerraIncognita | accuracy: 47.4 -> 48.1 ± 1.2 | Helped |
| use arithmetic meta-learning instead of Fish | ResNet50 · DomainNet | accuracy: 42.7 -> 41.5 ± 0.1 | Hurt |
| use arithmetic meta-learning instead of Fish | CAMELYON17 · CAMELYON17 | accuracy: 74.8 -> 76.6 | Helped |
| add global weight averaging with SWAD (ablation) | Arith · DomainBed average | accuracy: 65.0 -> 66.7 | Helped |
| enable inner-loop momentum and domain-specific sampling (ablation) | Arith · PACS | accuracy: 86.5 -> 85.7 | Hurt |
| use uniform sampling instead of domain-specific sampling (ablation) | Arith · PACS | accuracy: 85.9 -> 86.5 | No clear effect |
| increase the learning rate to 1.5 times its original value (ablation) | Arith with SGD outer loop · Office-Home | accuracy: 67.4 -> 67.7 | Helped |
| increase the learning rate to 1.5 times its original value (ablation) | Arith with Adam outer loop · Office-Home | accuracy: 69.4 -> 69.1 | Hurt |
| use Adam instead of SGD in the outer loop (ablation) | Arith · Office-Home | accuracy: 67.4 -> 69.4 | Helped |
Surprising findings
- Adam in the inner loop with momentum mixes domain gradients so that gradient matching resembles uniform sampling, motivating SGD without momentum inside the inner loop.
Limitations
- Built on first-order meta-learning, so it may fail to converge to flat minima.
- Balance among source domains may not translate to optimal performance on unseen target domains.
- Gains on some datasets (e.g., TerraIncognita) are within a few points with substantial variance.
Experimental setup
Datasets: PACS Office-Home VLCS TerraIncognita DomainNet CAMELYON17
Benchmarks: DomainBed PACS DomainBed VLCS DomainBed OfficeHome DomainBed TerraIncognita DomainBed DomainNet WILDS CAMELYON17
Metrics: accuracy
Baselines: ERM Fish MLDG Fishr CORAL SWAD MAML
Builds on: Fish MLDG Reptile SWAD
Closest papers by meaning
- Gradient Matching for Domain Generalization (ICLR 2022 · similarity 0.89)
- Learning to Generalize: Meta-Learning for Domain Generalization (AAAI 2018 · similarity 0.88)
- On the Stability and Generalization of Meta-Learning: the Impact of Inner-Levels (NeurIPS 2025 · similarity 0.87)
- One-Step Generalization Ratio Guided Optimization for Domain Generalization (ICML 2025 · similarity 0.87)
- Gradient-Guided Annealing for Domain Generalization (CVPR 2025 · similarity 0.85)
- Federated Domain Generalization with Data-free On-server Matching Gradient (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.