Research map / Label-efficient and robust learning / Semi-supervised learning
RegMixMatch: Optimizing Mixup Utilization in Semi-Supervised Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RegMixMatch, an SSL framework that applies RegMixup (Mixup plus clean samples) with pseudo-labels for high-confidence data and a class-aware Mixup with softmax targets for low-confidence data, reaching state-of-the-art error rates on CIFAR-10/100, SVHN and STL-10.
Problem
Mixup in semi-supervised learning increases prediction entropy and degrades artificial label purity, while confidence thresholding discards low-confidence unlabeled samples, underusing them.
Approach
Semi-supervised RegMixup keeps clean pseudo-labeled samples alongside ResizeMix-mixed high-confidence samples to preserve label purity, and class-aware Mixup pairs low-confidence samples with high-confidence samples of the same top predicted class, using softmax targets and an L2 loss.
Key results
- RegMixMatch reaches 4.35% error on CIFAR-10 with 10 labels and 11.74% on STL-10 with 40 labels, beating the second-best results by 3.72 and 3.71 points.
- Achieves state-of-the-art in 9 of 12 standard SSL settings, second-best in the remaining 3 (per the paper).
- On ImageNet with MAE pre-trained ViT-B, top-1 accuracy is 58.35% with 1 label per class vs 54.69% for FreeMatch; ablation on STL-10 40 labels shows removing clean samples adds +4.04 error.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use RegMixMatch instead of the strongest listed baseline | SSL model · CIFAR-10, 10 labels | error rate (%): 8.07 -> 4.35 | Helped |
| use RegMixMatch instead of the strongest listed baseline | SSL model · CIFAR-10, 40 labels | error rate (%): 4.80 -> 4.24 | Helped |
| use RegMixMatch instead of the strongest listed baseline | SSL model · CIFAR-10, 250 labels | error rate (%): 4.22 -> 4.21 | No clear effect |
| use RegMixMatch instead of the strongest listed baseline | SSL model · CIFAR-10, 4000 labels | error rate (%): 3.61 -> 3.38 | Helped |
| use RegMixMatch instead of the strongest listed baseline | SSL model · CIFAR-100, 400 labels | error rate (%): 37.10 -> 35.27 | Helped |
| use RegMixMatch instead of the strongest listed baseline | SSL model · CIFAR-100, 2500 labels | error rate (%): 25.38 -> 23.78 | Helped |
| use RegMixMatch instead of the strongest listed baseline | SSL model · CIFAR-100, 10000 labels | error rate (%): 19.01 -> 19.41 | Hurt |
| use RegMixMatch instead of the strongest listed baseline | SSL model · SVHN, 40 labels | error rate (%): 1.96 -> 1.81 | Helped |
| use RegMixMatch instead of the strongest listed baseline | SSL model · SVHN, 250 labels | error rate (%): 1.43 -> 1.77 | Hurt |
| use RegMixMatch instead of the strongest listed baseline | SSL model · SVHN, 1000 labels | error rate (%): 1.41 -> 1.79 | Hurt |
| use RegMixMatch instead of the strongest listed baseline | SSL model · STL-10, 40 labels | error rate (%): 15.45 -> 11.74 | Helped |
| use RegMixMatch instead of the strongest listed baseline | SSL model · STL-10, 1000 labels | error rate (%): 4.82 -> 4.66 | Helped |
Surprising findings
- Mixup reduces the purity of artificial labels during SSL training, lowering performance below ERM-based FixMatch in the authors' experiments.
- Top-2 accuracy over all unlabeled data exceeds the reliability of pseudo-labels, motivating use of low-confidence samples.
Limitations
- Mixing more than two samples gave no further gains, and top-n accuracy with n>=3 was not exploited.
- Low-confidence utilization is described as an initial exploration needing further study.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 SVHN STL-10 ImageNet EuroSAT
Benchmarks: CIFAR-10 CIFAR-100 SVHN STL-10 ImageNet EuroSAT
Metrics: error rate top-1 accuracy top-5 accuracy
Baselines: VAT Mean Teacher MixMatch ReMixMatch UDA FixMatch Dash MPL FlexMatch SoftMatch FreeMatch SequenceMatch
Builds on: FixMatch RegMixup FreeMatch ResizeMix MixMatch
Closest papers by meaning
- Scaling Up Semi-supervised Learning with Unconstrained Unlabelled Data (AAAI 2024 · similarity 0.89)
- OwMatch: Conditional Self-Labeling with Consistency for Open-World Semi-Supervised Learning (NeurIPS 2024 · similarity 0.87)
- Contrastive Credibility Propagation for Reliable Semi-supervised Learning (AAAI 2024 · similarity 0.87)
- CGMatch: A Different Perspective of Semi-supervised Learning (CVPR 2025 · similarity 0.87)
- Rethinking Confidence Scores and Thresholds in Pseudolabeling-based SSL (ICML 2025 · similarity 0.86)
- Erasing the Bias: Fine-Tuning Foundation Models for Semi-Supervised Learning (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.