Research map / Label-efficient and robust learning / Noisy labels
Understanding Domain Generalization: A Noise Robustness Perspective
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows theoretically and empirically that label noise worsens ERM's reliance on spurious correlations while some DG algorithms (IRM, V-REx, GroupDRO) are implicitly noise-robust, but this robustness does not translate into better real-world benchmark performance than ERM.
Problem
Despite many domain generalization algorithms, there is no clear evidence they outperform ERM on standard benchmarks; the paper asks whether DG algorithms benefit over ERM under label noise.
Approach
Frames label noise as an additional failure mode for ERM: with overparameterized linear models trained on finite noisy samples, min-norm bias makes memorizing noise push the classifier toward spurious features. It shows IRMv1 and V-REx gradients are rescaled ERM gradients whose coefficients turn negative at high confidence, which resists memorization, and tests this across synthetic and real bench
Key results
- On CMNIST with label noise, IRM and V-REx outperform ERM-based methods (ERM, Mixup) as noise increases, and ERM's accuracy deteriorates faster.
- On real-world benchmarks with 10% and 25% label noise, no DG algorithm consistently beats ERM; Mixup is best on domain-shift datasets (e.g., PACS 83.6 vs ERM 82.0 at 10% noise).
- Authors note that for Waterbirds and CelebA, ERM, IRM, and V-REx with group labels set new baseline records that significantly exceed prior reported scores.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase label noise during training | ERM · CMNIST | minority-group test accuracy | Hurt |
| Use IRM or V-REx instead of ERM-based methods as label noise increases | CMNIST · CMNIST | test accuracy | Helped |
| Train for longer with 25% label noise | IRM and V-REx · CMNIST | noisy-label memorization | No clear effect |
| Train for longer with 25% label noise | ERM, Mixup, and GroupDRO · CMNIST | noisy-label memorization | Hurt |
| Add label noise to training data | ERM, Mixup, GroupDRO, IRM, and V-REx · Real-world domain-shift datasets | cross-test accuracy | Hurt |
| Use Mixup instead of other tested algorithms | ResNet-50 pretrained on ImageNet · Real-world domain-shift datasets | cross-test accuracy | Helped |
| Use IRM or V-REx instead of ERM | ResNet-50 pretrained on ImageNet · Real-world subpopulation-shift and domain-shift datasets | OOD accuracy | Mixed |
| Increase label noise in the finite-sample setting | ERM with overparameterized models · Subpopulation shifts | minority-group generalization | Hurt |
Surprising findings
- Despite theoretical noise robustness, IRM and V-REx rarely catch up with ERM on real-world domain-shift datasets.
- Mixup, a simple augmentation, consistently outperforms other algorithms on real-world domain-shift datasets under label noise.
Experimental setup
Datasets: ColoredMNIST Waterbirds CelebA PACS VLCS OfficeHome TerraIncognita CivilComments ImageNet
Benchmarks: ColoredMNIST Waterbirds CelebA Waterbirds+ CelebA+ PACS VLCS OfficeHome TerraIncognita
Metrics: worst-group accuracy out-of-distribution accuracy test error noise memorization accuracy
Baselines: ERM Mixup GroupDRO IRM V-REx
Builds on: IRM V-REx GroupDRO DomainBed Mixup
Closest papers by meaning
- Set Valued Predictions For Robust Domain Generalization (ICML 2025 · similarity 0.91)
- Robust Domain Generalization under Divergent Marginal and Conditional Distributions (NeurIPS 2026 · similarity 0.89)
- Label Noise: Ignorance Is Bliss (NeurIPS 2024 · similarity 0.89)
- Lost Domain Generalization Is a Natural Consequence of Lack of Training Domains (AAAI 2024 · similarity 0.88)
- The Price of Implicit Bias in Adversarially Robust Generalization (NeurIPS 2024 · similarity 0.88)
- Rethinking Multi-domain Generalization with A General Learning Objective (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.