Research map / Label-efficient and robust learning / Recognition and long-tail
Selective Mixup Helps with Distribution Shifts, But Not (Only) because of Mixup
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that selective mixup's gains on distribution-shift benchmarks are largely explained by an implicit resampling effect (regression toward the mean in class/domain distributions) rather than by mixing, and that simple resampling matches or combines well with it.
Problem
Selective mixup methods (mixing only selected pairs, e.g. across classes or domains) report strong out-of-distribution gains, but their mechanisms and limitations are poorly understood.
Approach
Selecting pairs non-randomly changes the training distribution: for 'different class' selection in binary tasks, the resulting distribution is uniform over classes, and more generally it moves class/domain distributions toward uniformity. A 'selective sampling' ablation (selected pairs without mixing) reproduces much of the gain, showing resampling rather than mixing drives the improvement.
Key results
- On Waterbirds, the best selective mixup variant's gains are fully matched by resampling; selective sampling without mixing reproduces the effect.
- On MIMIC-Readmission, resampling for uniform classes plus concatenated pairs (no mixing) gives nearly the same performance as with vanilla mixup, and the sampled majority-class proportion (64.3%) matches the test proportion (66.5%) better than the training data (78.2%).
- On Yearbook, divergence reduction between training and test class distributions correlates strongly with test accuracy, and combining uniform-class resampling with same-class selective mixup gives results superior to all prior results reported in the paper.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the different-class selective sampling criterion without mixing (ablation) | Selective mixup · Waterbirds | worst-group accuracy | Helped |
| Replace mixing with selective sampling (append selected pairs without mixing) (ablation) | Selective mixup · Yearbook | worst-domain accuracy | Helped |
| Use uniform-class resampling | ERM · Yearbook | worst-domain accuracy | Helped |
| Use uniform-class resampling | ERM · arXiv | OOD accuracy | Hurt |
| Replace mixing with selective sampling (append selected pairs without mixing) (ablation) | Selective mixup · arXiv | OOD accuracy | Helped |
| Combine vanilla mixup with resampling | ERM · arXiv | OOD accuracy | Helped |
| Use vanilla mixup | ERM · arXiv | OOD accuracy | Helped |
| Use uniform-class resampling | ERM · MIMIC-Readmission | worst-domain AUROC | Helped |
| Add vanilla mixup to uniform-class resampling (ablation) | ERM · MIMIC-Readmission | worst-domain AUROC | No clear effect |
| Swap the ID and OOD data, reversing the training-to-test label shift | Methods that improved OOD performance on Yearbook · Yearbook-reversed | OOD accuracy | Hurt |
| Add uniform-class sampling to same-class selective mixup | Selective mixup · Yearbook | worst-domain accuracy | Helped |
| Use vanilla mixup instead of ERM | ERM · Yearbook | worst-domain accuracy | Hurt |
Surprising findings
- Selective sampling without any mixing often matches the gains attributed to selective mixup, so the mixing itself is not the main driver in several benchmarks.
- Reversing the Yearbook ID/OOD split makes the methods that previously helped OOD performance become detrimental, indicating the benefit depends on a regression-toward-the-mean property of the data.
Limitations
- Only the simplest selective mixup (LISA-style pair selection) is studied; methods with additional objective regularizers are not analyzed.
- Only five datasets evaluated, all previously reported to benefit from selective mixup, so the regression-toward-the-mean condition could not be broadly tested.
- Uses small architectures (frozen features, BERT-tiny, bag of embeddings) rather than state-of-the-art models.
Experimental setup
Datasets: Waterbirds CivilComments Wild-Time Yearbook Wild-Time arXiv Wild-Time MIMIC-Readmission
Benchmarks: Waterbirds CivilComments Wild-Time Yearbook Wild-Time arXiv Wild-Time MIMIC-Readmission
Metrics: worst-group accuracy worst-domain accuracy AUROC in-domain accuracy OOD average accuracy
Baselines: ERM vanilla mixup uniform class resampling LISA SelecMix DomainMix
Builds on: mixup LISA importance-weighted ERM CutMix
Closest papers by meaning
- Few-shot Adaptation to Distribution Shifts By Mixing Source and Target Embeddings (ICML 2024 · similarity 0.86)
- MixMin: Finding Data Mixtures via Convex Minimization (ICML 2025 · similarity 0.86)
- Tailoring Mixup to Data for Calibration (ICLR 2025 · similarity 0.86)
- Mixup-Induced Domain Extrapolation for Domain Generalization (AAAI 2024 · similarity 0.86)
- What Does It Take to Build a Performant Selective Classifier? (NeurIPS 2025 · similarity 0.85)
- Pushing Boundaries: Mixup's Influence on Neural Collapse (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.