Research map / Label-efficient and robust learning / Self-supervised and contrastive
Towards domain-invariant Self-Supervised Learning with Batch Styles Standardization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Batch Styles Standardization (BSS), a Fourier-based batch-level style harmonization that plugs into SimCLR, SWaV and MSN to improve unsupervised domain generalization without domain labels.
Problem
Self-supervised models generalize poorly to unseen domains; existing unsupervised domain generalization (UDG) methods need domain labels and domain-specific architectures that scale poorly.
Approach
Standardize image style within a batch by transferring low-frequency Fourier amplitude of a randomly chosen image to all batch images, so positives and negatives share a pseudo-domain; this reduces spurious domain correlations in contrastive and non-contrastive SSL without domain labels.
Key results
- On PACS with 1% labels, SimCLR w/ BSS improves over SimCLR w/ FA by +2.96% average accuracy and SWaV w/ BSS by +5.3%.
- On Camelyon17 WILDS, SimCLR w/ BSS reaches 92.27% at 1% labels and 95.00% at 100% labels, versus 90.82% and 93.44% for SimCLR w/ FA.
- On DomainNet, SWaV w/ BSS improves average per-domain accuracy by +2.08% (1% labels) over SWaV w/ FA, and matches or beats several UDG methods such as BrAD at some label fractions.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add BSS to SimCLR instead of FA | SimCLR · PACS, avg. accuracy across label fractions | accuracy: +2.96%, +4.55%, +3.61%, +3.6% | Helped |
| Add BSS to SWaV instead of FA | SWaV · PACS, avg. accuracy across label fractions | accuracy: +5.3%, +5.9%, +5.46%, +1.52% | Helped |
| Add BSS instead of FA | SimCLR · DomainNet, per-domain average accuracy | accuracy: +1.51%, +1.16%, +1.33% | Helped |
| Add BSS instead of FA | SWaV · DomainNet, per-domain average accuracy | accuracy: +2.08%, +1.61%, +1.11% | Helped |
| Add BSS instead of FA | SimCLR, SWaV, and MSN · Camelyon17 WILDS test | accuracy: +1.45% to +4.49% | Helped |
| Change color jitter from sample-wise to batch-wise (ablation) | SimCLR without FA or BSS · Camelyon17 WILDS | accuracy: 63.54, 63.58, 63.50, 65.06 -> 66.81, 68.19, 68.45, 68.53 | Helped |
| Add FA with sample-wise color jitter (ablation) | SimCLR · Camelyon17 WILDS | accuracy: 63.54, 63.58, 63.50, 65.06 -> 90.82, 93.00, 92.94, 93.44 | Helped |
| Use BSS with batch-wise color jitter instead of sample-wise color jitter with FA (ablation) | SimCLR · Camelyon17 WILDS | accuracy: 90.82, 93.00, 92.94, 93.44 -> 92.27, 94.75, 94.82, 95.00 | Helped |
| Add BSS instead of FA | SWaV · DomainNet, target infograph, 1% labels | accuracy: 20.95 -> 20.12 | Hurt |
| Add BSS instead of FA | SimCLR · DomainNet, target real, 5% labels | accuracy: 41.95 -> 41.49 | Hurt |
| Add BSS instead of FA | SWaV · DomainNet, target infograph and sketch, 10% labels | accuracy: 24.39 -> 24.34; 60.78 -> 60.55 | Mixed |
Surprising findings
- Batch-wise color jitter alone (no FA or BSS) gives a notable gain over sample-wise jitter, suggesting reduced style variability in a batch helps generalization.
- On PACS, several UDG baselines (BrAD, DiMAE, CycleMAE) that use ImageNet transfer learning beat BSS on the photo domain, which the authors attribute to ImageNet overlap.
Limitations
- Relies on ImageNet transfer learning for some comparisons on DomainNet and reimplemented UDG baselines; fairness issues noted by authors.
- Evaluated only on three UDG benchmarks with a limited class subset for DomainNet.
- Code stated to be released only upon acceptance; not yet available in the paper.
Experimental setup
Datasets: PACS DomainNet Camelyon17 WILDS ImageNet
Benchmarks: PACS DomainNet Camelyon17 WILDS
Metrics: accuracy domain purity prototype homogeneity cosine similarity of anchor-negative pairs
Baselines: MoCo V2 SimCLR V2 BYOL AdCo MAE DARLING DiMAE BrAD CycleMAE FixMatch SWaV (WILDS challenge)
Builds on: Fourier-based Augmentations SimCLR SWaV MSN BrAD DARLING
Closest papers by meaning
- Is Large-scale Pretraining the Secret to Good Domain Generalization? (ICLR 2025 · similarity 0.87)
- Towards Generalizing to Unseen Domains with Few Labels (CVPR 2024 · similarity 0.87)
- START: A Generalized State Space Model with Saliency-Driven Token-Aware Transformation (NeurIPS 2024 · similarity 0.86)
- Unknown Domain Inconsistency Minimization for Domain Generalization (ICLR 2024 · similarity 0.86)
- Minimal Semantic Sufficiency Meets Unsupervised Domain Generalization (NeurIPS 2025 · similarity 0.86)
- Unlocking the Potential of Unlabeled Data in Semi-Supervised Domain Generalization (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.