Research map / Optimization / Optimizers
Domain-Inspired Sharpness-Aware Minimization Under Domain Shifts
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DISAM, a SAM variant that adds a domain-loss variance term to the sharpness-estimation objective so perturbations are calibrated by per-domain convergence, improving domain generalization on DomainBed and CLIP-based PEFT.
Problem
SAM's sharpness estimation assumes i.i.d. data; under domain shifts, inconsistent convergence across domains biases the perturbation toward less-converged domains, degrading generalization (e.g., SAM worse than ERM on TerraInc).
Approach
Add a variance-of-domain-losses regularizer to the loss used for perturbation generation, so gradient perturbations are automatically weakened toward domains with above-average loss and strengthened toward well-optimized domains; plugs into SAM, GSAM, SAGM, and other methods with negligible overhead.
Key results
- With ResNet50 on DomainBed, DISAM+SAGM reaches 67.0 average out-of-domain accuracy vs 66.1 for SAGM, and DISAM+SAM improves over SAM by 1.9 points on average.
- With CLIP ViT-B/16, CLIPOOD+DISAM achieves 78.8 average accuracy vs 77.9 for CLIPOOD*, and CoOp+DISAM reaches 74.8 vs 74.2 with SAM.
- On the open-class OfficeHome/DomainNet split, DISAM improves total accuracy by 3.3% over CoOp and 3.1% over CLIPOOD.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add DISAM to SAM | SAM with ResNet50 · DomainBed five-dataset average, out-of-domain | accuracy: 64.5 -> 66.4; average improvements of 1.9% | Helped |
| Add DISAM to GSAM | GSAM with ResNet50 · DomainBed five-dataset average, out-of-domain | accuracy: 65.1 -> 66.8; average improvements of 1.7% | Helped |
| Add DISAM to SAGM | SAGM with ResNet50 · DomainBed five-dataset average, out-of-domain | accuracy: 66.1 -> 67.0; average improvements of 1.9% | Helped |
| Add DISAM to CoOp | CoOp with CLIP ViT-B/16 · DomainBed five-dataset average, out-of-domain | accuracy: 74.2 -> 74.8 | Helped |
| Replace SAM with DISAM on CLIPOOD* | CLIPOOD* with CLIP ViT-B/16 · DomainBed five-dataset average, out-of-domain | accuracy: 76.9 -> 78.8 | Helped |
| Add DISAM to CoOp* | CoOp* with CLIP · OfficeHome and DomainNet, open classes, Total | accuracy: 71.5 -> 72.2 | Helped |
| Replace SAM with DISAM on CLIPOOD* | CLIPOOD* with CLIP · OfficeHome and DomainNet, open classes, Total | accuracy: 71.8 -> 72.6 | Helped |
| Add DISAM to V-REx | V-REx · PACS | accuracy: 84.9 -> 85.8 | Helped |
| Add DISAM to Fishr | Fishr · PACS | accuracy: 86.9 -> 87.5 | Helped |
| Use too large or too small perturbation amplitude rho (ablation) | DISAM · PACS and OfficeHome | accuracy | Hurt |
| Increase variance constraint weight lambda beyond the stable range (ablation) | DISAM · PACS and OfficeHome | accuracy | Hurt |
| Add domain-inspired perturbation adjustment (ablation) | SAM · PACS, unseen test domain | gradient variance | Helped |
Surprising findings
- Vanilla SAM performs worse than ERM on TerraIncognita (43.3 vs 46.1 out-of-domain with ResNet50), showing SAM can be harmful under domain shifts.
- SAM also degrades CLIPOOD* on several DomainNet/OfficeHome splits relative to the non-SAM baseline.
Limitations
- Convergence theorem relies on approximations (e.g., assumed negligible inner-product expectation and ad hoc Gamma bound) that are not rigorously justified.
- Gains on CLIP-based adaptation are marginal on some datasets, attributed to early stopping limiting overfitting.
- Sensitivity to lambda: too large values increase variance across repeated runs.
Experimental setup
Datasets: PACS VLCS OfficeHome TerraIncognita DomainNet ImageNet
Benchmarks: PACS VLCS OfficeHome TerraIncognita DomainNet
Metrics: accuracy
Baselines: ERM SAM GSAM SAGM CORAL CoOp CLIPOOD V-REx Fishr
Builds on: Sharpness-Aware Minimization GSAM SAGM V-REx Fishr
Closest papers by meaning
- Unknown Domain Inconsistency Minimization for Domain Generalization (ICLR 2024 · similarity 0.88)
- Explicit Eigenvalue Regularization Improves Sharpness-Aware Minimization (NeurIPS 2024 · similarity 0.87)
- A Universal Class of Sharpness-Aware Minimization Algorithms (ICML 2024 · similarity 0.87)
- Seeking Consistent Flat Minima for Better Domain Generalization via Refining Loss Landscapes (CVPR 2025 · similarity 0.87)
- Zeroth-Order Sharpness-Aware Learning with Exponential Tilting (NeurIPS 2026 · similarity 0.87)
- Lookbehind-SAM: k steps back, 1 step forward (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.