Research map / Label-efficient and robust learning / Domain adaptation and generalization

Gradient-Guided Annealing for Domain Generalization

CVPR 2025 · Label-efficient and robust learning › Domain adaptation and generalization · approach: Optimization-based domain generalization · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Gradient-Guided Annealing (GGA), which during early training randomly perturbs parameters and keeps perturbations that increase minimum pairwise cross-domain gradient similarity without raising loss, improving ResNet-50 domain generalization on five DomainBed benchmarks.

Problem

Domain generalization fails because gradients from different source domains conflict during training, driving optimization to local minima that rely on domain-specific features rather than domain-invariant ones.

Approach

Inspired by simulated annealing, GGA runs a short search in early training that perturbs parameters with uniform noise and accepts points that raise the minimum pairwise domain gradient cosine similarity while keeping loss within a small margin, then continues standard optimization; it can also be stacked on existing DG methods.

Key results

• GGA raises ERM average out-of-domain accuracy from 63.9 to 66.3 across five DG benchmarks (ResNet-50, DomainBed).
• GGA reaches 87.3 on PACS and 50.6 on TerraIncognita, setting state-of-the-art on these two datasets per the paper.
• Combining GGA with prior DG methods improves average accuracy in most cases, by about 1% on average and up to 6.3% (MMD on TerraIncognita).

Measured effects reported in the paper

Surprising findings

• IRM is negatively affected by GGA (average -2.1), which the authors attribute to its objective not targeting pairwise gradient agreement.

Limitations

• Adds computational overhead from annealing steps and gradient similarity computation.
• Requires domain labels, so it is not applicable to single-source DG.
• Effectiveness depends on batch size, since larger batches give better gradient alignment estimates.

Experimental setup

Datasets: PACS VLCS OfficeHome TerraIncognita DomainNet

Benchmarks: PACS VLCS OfficeHome TerraIncognita DomainNet

Metrics: top-1 out-of-domain accuracy

Baselines: ERM Mixstyle GroupDRO MMD AND-mask ARM IRM MTL VREx MLDG Mixup SagNet

Builds on: ERM Fish SAM DomainBed Simulated Annealing

Closest papers by meaning

• One-Step Generalization Ratio Guided Optimization for Domain Generalization (ICML 2025 · similarity 0.90)
• Seeking Consistent Flat Minima for Better Domain Generalization via Refining Loss Landscapes (CVPR 2025 · similarity 0.89)
• Rethinking Multi-domain Generalization with A General Learning Objective (CVPR 2024 · similarity 0.89)
• Sharpness-Aware Model-Agnostic Long-Tailed Domain Generalization (AAAI 2024 · similarity 0.88)
• Unknown Domain Inconsistency Minimization for Domain Generalization (ICLR 2024 · similarity 0.88)
• PracticalDG: Perturbation Distillation on Vision-Language Models for Hybrid Domain Generalization (CVPR 2024 · similarity 0.88)