Research map / Label-efficient and robust learning / Semi-supervised learning
Scaling Up Semi-supervised Learning with Unconstrained Unlabelled Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes UnMixMatch, a semi-supervised method that learns from unconstrained unlabelled data (ImageNet subsets with different classes/distribution) by combining RandMixUp supervised learning, a contrastive consistency regularizer on embeddings, and rotation prediction.
Problem
Semi-supervised learning typically assumes labelled and unlabelled data share classes and distribution; pseudo-label methods break when unlabelled data contain unrelated classes, limiting scaling to web-scale unlabelled data.
Approach
Use hard augmentation (RandAug + MixUp, called RandMixUp) for the supervised loss, enforce consistency between two strongly augmented views in the embedding space via InfoNCE rather than on class predictions, and add rotation prediction as a self-supervised pretext task on unlabelled data.
Key results
- Average accuracy 68.77% across 10 unconstrained settings, versus 63.98% for the next best (CCSSL), a reported ~4.79% gain.
- On CIFAR-10 with 40 labels, accuracy rises from 47.93% to 53.54% when the unlabelled set grows from ImageNet-100 (130K) to ImageNet-1K (1.28M).
- On open set CIFAR-10 (6/4 split) with 50 labels per class, reaches 95.7% versus 89.6% for OpenMatch.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use UnMixMatch instead of CCSSL | Unconstrained semi-supervised learning · Average across CIFAR-10, CIFAR-100, SVHN, and STL-10 settings | accuracy: 68.77 vs 63.98 | Helped |
| Increase unlabelled data from ImageNet-100 to ImageNet-1K | UnMixMatch · CIFAR-10, 40 labels | accuracy: 47.93% to 53.54%; 5.61% | Helped |
| Add RandMixUp to the supervised module (ablation) | UnMixMatch · CIFAR-10, 40 labels | accuracy: 38.72 -> 47.93; 9.21% | Helped |
| Add contrastive consistency loss (ablation) | UnMixMatch · CIFAR-10, 40 labels | accuracy: 41.25 -> 47.93; 6.68% | Helped |
| Add rotation prediction loss (ablation) | UnMixMatch · CIFAR-10, 40 labels | accuracy: 41.83 -> 47.93; 6.1% | Helped |
| Replace RandMixUp with RandAug (ablation) | UnMixMatch · CIFAR-10, 40 labels | accuracy: 47.93 -> 46.01 | Hurt |
| Replace RandMixUp with MixUp (ablation) | UnMixMatch · CIFAR-10, 40 labels | accuracy: 47.93 -> 45.23; 2.7% decrease | Hurt |
| Replace RandMixUp with CutMix (ablation) | UnMixMatch · CIFAR-10, 40 labels | accuracy: 47.93 -> 43.12 | Hurt |
| Replace RandMixUp with AugMix (ablation) | UnMixMatch · CIFAR-10, 40 labels | accuracy: 47.93 -> 44.75 | Hurt |
| Replace the proposed contrastive loss with ConMatch (ablation) | UnMixMatch · CIFAR-10, 40 labels | accuracy: 47.93 -> 45.82; 2.16% | Hurt |
| Replace the proposed contrastive loss with Contrastive Regularization (ablation) | UnMixMatch · CIFAR-10, 40 labels | accuracy: 47.93 -> 45.77; 2.11% | Hurt |
| Replace the proposed contrastive loss with Graph Contrast (ablation) | UnMixMatch · CIFAR-10, 40 labels | accuracy: 47.93 -> 47.12; 0.81% | Hurt |
Surprising findings
- Existing SSL methods often perform worse with unconstrained unlabelled data than in standard settings, with ReMixMatch and CoMatch showing only small gains from more unlabelled data.
- Replacing the contrastive loss with VICReg gives competitive results (67.85% average vs 68.77%).
Limitations
- Hard augmentations and rotation prediction may need tuning for specialized domains.
- Evaluated only on image classification benchmarks with small-resolution datasets.
- Reported results are averaged over three runs (five for barely supervised), with hyperparameters closely following FlexMatch.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 SVHN STL-10 ImageNet-100 ImageNet-1K
Benchmarks: CIFAR-10 CIFAR-100 SVHN STL-10 CIFAR-10 open set (6/4 split)
Metrics: accuracy average accuracy across settings
Baselines: Supervised Pi-Model Mean Teacher VAT Pseudo-label UDA MixMatch ReMixMatch FixMatch FlexMatch CoMatch CCSSL
Builds on: FlexMatch ReMixMatch MixMatch FixMatch CCSSL
Closest papers by meaning
- InterLUDE: Interactions between Labeled and Unlabeled Data to Enhance Semi-Supervised Learning (ICML 2024 · similarity 0.90)
- CGMatch: A Different Perspective of Semi-supervised Learning (CVPR 2025 · similarity 0.90)
- OwMatch: Conditional Self-Labeling with Consistency for Open-World Semi-Supervised Learning (NeurIPS 2024 · similarity 0.89)
- Unlocking the Potential of Unlabeled Data in Semi-Supervised Domain Generalization (CVPR 2025 · similarity 0.89)
- RegMixMatch: Optimizing Mixup Utilization in Semi-Supervised Learning (AAAI 2025 · similarity 0.89)
- Out-Of-Domain Unlabeled Data Improves Generalization (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.