Research map / Label-efficient and robust learning / Domain adaptation and generalization
Minimal Semantic Sufficiency Meets Unsupervised Domain Generalization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Formalizes unsupervised domain generalization as learning a minimal sufficient semantic representation and implements it as MS-UDG, combining InfoNCE with a mixed-InfoNCE disentanglement loss and a reconstruction loss, without domain or class labels.
Problem
Unsupervised domain generalization: learning self-supervised representations that generalize to unseen domains without category or domain labels, separating semantic information from domain-specific variation.
Approach
Defines a minimal sufficient semantic representation via information theory and optimizes it with a mixed InfoNCE loss (minimizing mutual information between semantic and variation parts) plus a cross-reconstruction loss (maximizing variation information about the input), built on Fourier-augmented contrastive learning.
Key results
- On PACS with 1% labels, MS-UDG reaches 45.26% average accuracy versus 44.91% for the second-best BSS/SimCLR.
- On PACS with 10% labels, MS-UDG reaches 68.35% average accuracy, a reported +5.48% over CycleMAE in the text's comparison.
- On DomainNet with 1% labels, MS-UDG reaches 32.45% overall accuracy versus 32.27% for BSS/SimCLR; with 10% labels it reaches 53.37% overall versus 50.78% for CycleMAE.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MS-UDG instead of BSS/SimCLR | UDG models with ViT-S/16 · PACS, 1% label fraction | macro-accuracy: +0.35% | Helped |
| Use MS-UDG instead of BSS/SimCLR | UDG models with ViT-S/16 · PACS, 5% label fraction | macro-accuracy: +6.62% | Helped |
| Use MS-UDG instead of CycleMAE | UDG models with ViT-S/16 · PACS, 10% label fraction | macro-accuracy: +5.48% | Helped |
| Use MS-UDG instead of CycleMAE | UDG models with ViT-S/16 · PACS, 100% label fraction | macro-accuracy: +1.48% | Helped |
| Use MS-UDG instead of BSS/SimCLR | UDG models with ViT-S/16 · DomainNet, 1% label fraction | overall accuracy: +0.18% | Helped |
| Use MS-UDG instead of BSS/SimCLR | UDG models with ViT-S/16 · DomainNet, 5% label fraction | overall accuracy: +1.6% | Helped |
| Use MS-UDG instead of CycleMAE | UDG models with ViT-S/16 · DomainNet, 10% label fraction | overall accuracy: +2.59% | Helped |
| Add L_min to the L_suff baseline (ablation) | MS-UDG · PACS, 1% label fraction | accuracy: 42.45 -> 42.85 | Helped |
| Add L_min to the L_suff baseline (ablation) | MS-UDG · PACS, 5% label fraction | accuracy: 56.17 -> 57.91 | Helped |
| Add L_max to the L_suff baseline (ablation) | MS-UDG · PACS, 10% label fraction | accuracy: 64.81 -> 67.40 | Helped |
| Add L_min and L_max to the L_suff baseline (ablation) | MS-UDG · PACS, 1% label fraction | accuracy: +2.81% | Helped |
| Add L_min and L_max to the L_suff baseline (ablation) | MS-UDG · PACS, 5% label fraction | accuracy: 6.85% | Helped |
Surprising findings
- On the PACS 1% label setting, the ablation shows the L_min term alone yields only a small gain (42.45 to 42.85), while L_max alone yields a larger one (to 44.77).
Limitations
- Primarily tailored for scenarios with pronounced domain discrepancies; may be less effective under subtle covariate shifts.
- Theoretical guarantees rely on approximations (e.g., Bayes error bound and the downstream-task assumption I(x1;x2;T)=I(x1;x2;S)).
Experimental setup
Datasets: PACS DomainNet
Benchmarks: PACS DomainNet
Metrics: accuracy macro-accuracy micro-accuracy (overall)
Baselines: ERM MoCo V2 SimCLR V2 BYOL AdCo MAE DARLING DiMAE BrAD BSS/SimCLR CycleMAE
Builds on: InfoNCE Fourier-based augmentation (FDA) DARLING DisMAE BSS
Closest papers by meaning
- Towards Generalizing to Unseen Domains with Few Labels (CVPR 2024 · similarity 0.91)
- Unknown Domain Inconsistency Minimization for Domain Generalization (ICLR 2024 · similarity 0.90)
- Multimodal Unsupervised Domain Generalization by Retrieving Across the Modality Gap (ICLR 2025 · similarity 0.89)
- Reasoning-Driven Multimodal LLM for Domain Generalization (ICLR 2026 · similarity 0.87)
- PracticalDG: Perturbation Distillation on Vision-Language Models for Hybrid Domain Generalization (CVPR 2024 · similarity 0.87)
- AdvST: Revisiting Data Augmentations for Single Domain Generalization (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.