Research map / Optimization / Optimizers
One-Step Generalization Ratio Guided Optimization for Domain Generalization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GENIE, a preconditioned optimizer that equalizes parameter-wise One-Step Generalization Ratio (OSGR) for domain generalization, with noise injection and random dropout masks, and reports gains over Adam/SGD/SAM on DomainBed.
Problem
Domain generalization models overfit to domain-specific spurious correlations, and existing gradient-based and sharpness-aware optimizers leave parameter updates imbalanced, with a few parameters dominating optimization.
Approach
Uses the One-Step Generalization Ratio (OSGR) and GSNR to derive a per-parameter preconditioning factor that equalizes parameter contributions to OSGR, plus gradient noise injection and random Bernoulli masking, yielding a new optimizer GENIE.
Key results
- GENIE reaches 66.9% average accuracy across five DG datasets with ResNet-50 under DomainBed, versus 63.3% for Adam and 62.9% for SGD (table in paper).
- Integrated with RSC and CORAL, GENIE raises averages to 71.4% and 71.9% from about 68% for the baseline optimizers.
- In the SDG setting, GENIE reaches 58.5% average accuracy versus 51.2% for Adam and 52.6% for SAM.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GENIE instead of FAD | ResNet-50 DG training · PACS | accuracy: 88.2 -> 87.8 | Hurt |
| Use GENIE instead of FAD | ResNet-50 DG training · VLCS | accuracy: 78.9 -> 80.7 | Helped |
| Use GENIE instead of FAD | ResNet-50 DG training · OfficeHome | accuracy: 69.2 -> 69.7 | Helped |
| Use GENIE instead of FAD | ResNet-50 DG training · TerraIncognita | accuracy: 45.7 -> 52.0 | Helped |
| Use GENIE instead of AdaHessian | ResNet-50 DG training · DomainNet | accuracy: 44.4 -> 44.1 | Hurt |
| Use GENIE instead of FAD | ResNet-50 DG training · Average across five DG datasets | accuracy: 65.3 -> 66.9 | Helped |
| Add preconditioning to ERM with Adam (ablation) | ERM with Adam on PACS · PACS | average accuracy: 84.2 -> 87.4 | Helped |
| Add noise injection to preconditioning without random mask (ablation) | GENIE components on PACS · PACS | average accuracy: 87.4 -> 85.0 | Hurt |
| Add random mask to preconditioning without noise injection (ablation) | GENIE components on PACS · PACS | average accuracy: 87.4 -> 85.1 | Hurt |
| Add noise injection and random mask to preconditioning (ablation) | GENIE components on PACS · PACS | average accuracy: 87.4 -> 88.4 | Helped |
Surprising findings
- SAM exhibits lower OSGR than GENIE despite being designed for generalization, which the authors take to mean sharpness alone is insufficient.
- Conventional optimizers (Adam, SGD) reduced performance when used with RSC in the single-domain setting, whereas GENIE improved it.
Limitations
- The theoretical derivations rely on approximations (dropping 1/n terms, Gaussian and Chebyshev-based bounds, uniform weights), and the paper notes the uniform-weight assumption can be overly restrictive.
- Several results lack variance across runs in the main table, and some baselines are reproduced from other papers.
- Ablation shows noise injection and random mask do not uniformly help; the authors say preconditioning alone is sufficient, with the others optional.
Experimental setup
Datasets: PACS VLCS OfficeHome TerraIncognita DomainNet FashionMNIST
Benchmarks: PACS VLCS OfficeHome TerraIncognita DomainNet
Metrics: accuracy average accuracy across domains training time per iteration OSGR
Baselines: SGD Adam AdamW AdaBelief AdaHessian YOGI SAM GAM FAD MIRO Mixstyle
Builds on: Adam SVAG SAM DomainBed OSGR (one-step generalization ratio)
Closest papers by meaning
- Gradient-Guided Annealing for Domain Generalization (CVPR 2025 · similarity 0.90)
- Unknown Domain Inconsistency Minimization for Domain Generalization (ICLR 2024 · similarity 0.89)
- Rethinking Multi-domain Generalization with A General Learning Objective (CVPR 2024 · similarity 0.89)
- Improving Domain Generalization with Domain Relations (ICLR 2024 · similarity 0.87)
- Robust Domain Generalization under Divergent Marginal and Conditional Distributions (NeurIPS 2026 · similarity 0.87)
- Is Large-scale Pretraining the Secret to Good Domain Generalization? (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.