Research map / Label-efficient and robust learning / Domain adaptation and generalization
Mixture and distillation for domain shift
Handles domain shift via mixtures of experts, diversification and data mixture optimization.
78 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Domain adaptation and generalization, keyword cluster “cross domain · transfer · domain shot”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
31 papers in 2023–24 and 19 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 9.1% to 6.0%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Multi-Domain Active Learning for Recommendation (AAAI 2016 · 52 citations)
- 2018 · Modeling Sparse Deviations for Compressed Sensing using Generative Models (ICML 2018 · 43 citations)
- 2019 · Learning Robust Global Representations by Penalizing Local Predictive Power (NeurIPS 2019 · 271 citations)
- 2020 · LTF: A Label Transformation Framework for Correcting Label Shift (ICML 2020 · 16 citations)
- 2021 · Deep Transfer Tensor Decomposition with Orthogonal Constraint for Recommender Systems (AAAI 2021 · 43 citations)
- 2022 · MultiplexNet: Towards Fully Satisfied Logical Constraints in Neural Networks (AAAI 2022 · 33 citations)
- 2023 · Cross-Domain Adaptative Learning for Online Advertisement Customer Lifetime Value Prediction (AAAI 2023 · 14 citations)
- 2024 · Model Reprogramming: Resource-Efficient Cross-Domain Machine Learning (AAAI 2024 · 27 citations)
- 2025 · MTL-LoRA: Low-Rank Adaptation for Multi-Task Learning (AAAI 2025 · 17 citations)
- 2026 · The Geometry of Sequential Learning: Lie-Bracket Prediction of Transfer Order (ICML 2026 · 0 citations)
Earliest papers here
- Multi-Domain Active Learning for Recommendation (AAAI 2016 · 52 citations)
- Modeling Sparse Deviations for Compressed Sensing using Generative Models (ICML 2018 · 43 citations)
- JointGAN: Multi-Domain Joint Distribution Learning with Generative Adversarial Nets (ICML 2018 · 24 citations)
What has been tried
64 papers here report 643 measured changes (a component added, removed or swapped, compared with a baseline): helped 67%, no clear effect 6%, hurt 22%, mixed 5%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| add ControlNet structure guidance | DomainNet Real→Painting · Accuracy (%) | 38.64 -> 40.29 | Helped | Domain Gap Embeddings for Generative Dataset Augmentation (CVPR 2024 · 9 citations) |
| add BN conversion to FT | CIFAR-10 ID · accuracy (%) | 97.66 -> 97.59 | Hurt | Domain-Aware Fine-Tuning: Enhancing Neural Network Adaptability (AAAI 2024 · 7 citations) |
| use SF-MAP instead of supervised learning across tested backbones | STL10 → CIFAR10 · target accuracy | 50.3 -> 15.7; 61 -> 21.4; 58.9 -> 36.9; 36.2 -> 20.7; 38 -> 24.6; 72.5 -> 47; 39.2 -> 28.1 | Mixed | MAP: MAsk-Pruning for Source-Free Model Intellectual Property Protection (CVPR 2024 · 7 citations) |
| Add DAL to the baseline | N-Caltech101 · accuracy | 79.66% -> 82.07% | Helped | An Efficient Knowledge Transfer Strategy for Spiking Neural Networks from Static to Event Domain (AAAI 2024 · 6 citations) |
| use restricted local negations instead of global negations | Old American West · average score | 2.98 -> 3.57 | Helped | NaRuto: Automatically Acquiring Planning Models from Narrative Texts (AAAI 2024 · 6 citations) |
| Add the fine-tuning phase | DomainNet · Avg. Acc. | 75.9 -> 77.3 | Helped | Decoupled Training: Return of Frustratingly Easy Multi-Domain Learning (AAAI 2024 · 4 citations) |
| Add style consistency to CTS (S&C) | PACS · ECE | 5.09 -> 4.31 | Helped | Consistency-Guided Temperature Scaling Using Style and Content Information for Out-of-Domain Calibration (AAAI 2024 · 3 citations) |
| add type prompt words | TACRED K=1 · F1 (%) | 20.2 -> 22.5 | Helped | BayesPrompt: Prompting Large-Scale Pre-Trained Language Models on Few-shot Inference via Debiased Domain Abstraction (ICLR 2024 · 3 citations) |
| Add Generator G to Frozen CLIP + GM under ERM training | iWildCam OOD · F1 | 39.6 -> 32.5 | Hurt | Adapting to Distribution Shift by Visual Domain Prompt Generation (ICLR 2024 · 2 citations) |
| Add cross-transfer loss | sci-Plex3 · R² (All genes) | 0.660 -> 0.817 | Helped | Learning Cross-Domain Representations for Transferable Drug Perturbations on Single-Cell Transcriptional Responses (AAAI 2025 · 1 citation) |
| Set λ=0, removing likelihood-ratio regularization | Communities and Crime covariate-shift regression · Coverage | Hurt | Conformal Inference under High-Dimensional Covariate Shifts via Likelihood-Ratio Regularization (NeurIPS 2025 · 1 citation) | |
| add the CGML module | AMiner System, 2-way 1-shot · accuracy (%) | 84.36 -> 85.76 | Helped | When Sparse Graph Representation Learning Falls into Domain Shift: Data Augmentation for Cross-Domain Graph Meta-Learning (Student Abstract) (AAAI 2024 · 1 citation) |
What reviewers will expect
Datasets and baselines named in the 8 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Baselines
- DoReMi · 2 of 8 papers
- RegMix · 2 of 8 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 50 papers; from OpenAlex and the papers' affiliations).
- Chinese Academy of Sciences · 4 papers
- University of Chinese Academy of Sciences · 3 papers
- Tsinghua University · 3 papers
- Carnegie Mellon University · 3 papers
- Stanford University · 3 papers
- Google · 3 papers
- University of Pennsylvania · 3 papers
- Fudan University · 3 papers
- Australian National University · 2 papers
- Korea Advanced Institute of Science and Technology · 2 papers
Use it
Download BibTeX of 38 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Cross-domain few-shot with transformers (116 papers)
- Statistical transfer learning (79 papers)
- Cross-domain text and graph transfer (54 papers)
Elsewhere in the topic
- Conditional distribution alignment (162 papers)
- Feature alignment for unsupervised adaptation (105 papers)
- Optimization-based domain generalization (101 papers)
- Adversarial domain adaptation (95 papers)
- Geometric alignment and synthetic-to-real adaptation (93 papers)
- Source-free domain adaptation (92 papers)
- Adaptation for semantic segmentation (81 papers)
- Statistical alignment and hypothesis testing (80 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).