Research map / Label-efficient and robust learning / Semi-supervised learning
Enhancing Semi-Supervised Learning via Representative and Diverse Sample Selection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RDSS, a model-free sample selection method that picks a representative and diverse subset for annotation by minimizing a new alpha-MMD criterion with a Frank-Wolfe-style greedy solver (GKHR), improving FlexMatch and FreeMatch under tiny label budgets.
Problem
Choosing which unlabeled samples to annotate for semi-supervised learning under extremely low label budgets, where random, stratified, and AL/SSAL selection methods perform poorly or have high labeling overhead.
Approach
Defines alpha-MMD, a trade-off between MMD-based representativeness and average-kernel diversity controlled by alpha, and solves it with GKHR (generalized kernel herding without replacement), a one-shot, model-free selection performed before training; the paper gives generalization and convergence bounds.
Key results
- RDSS with FreeMatch reaches 95.05% on CIFAR-10 with 40 labels, versus 93.81% for the best baseline USL and 93.41% for random sampling.
- On STL-10 with 40 labels, RDSS with FreeMatch reaches 81.90% versus 76.95% for USL, a 4.95-point gain.
- On ImageNet with 100k selected labels, FreeMatch with RDSS achieves 58.24% versus 56.86% with USL.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use RDSS instead of USL for sample selection | FreeMatch · ImageNet-1k, 100000 labels | Top-1 accuracy: 56.86\% -> 58.24\% | Helped |
| Use RDSS instead of random sample selection | FlexMatch · CIFAR-10, 40 labels | Top-1 accuracy: 87.30 -> 94.69$\pm$0.28 | Helped |
| Use RDSS instead of random sample selection | FreeMatch · STL-10, 40 labels | Top-1 accuracy: 76.66 -> 81.90$\pm$1.72 | Helped |
| Use RDSS instead of random sample selection | FlexMatch · CIFAR-10, 40 labels | Top-1 accuracy: +7.39 | Helped |
| Use RDSS instead of random sample selection | FreeMatch · CIFAR-10, 40 labels | Top-1 accuracy: +1.64 | Helped |
| Add the representativeness term (instead of α=0) (ablation) | RDSS with FreeMatch · CIFAR-10, 40 labels | Top-1 accuracy: 85.54$\pm$0.48 -> 95.05$\pm$0.13 | Helped |
| Add the representativeness term (instead of α=0) (ablation) | RDSS with FreeMatch · CIFAR-100, 400 labels | Top-1 accuracy: 39.26$\pm$0.52 -> 48.41$\pm$0.59 | Helped |
| Add the diversity term (instead of α=1) (ablation) | RDSS with FreeMatch · CIFAR-10, 40 labels | Top-1 accuracy: 94.53$\pm$0.42 -> 95.05$\pm$0.13 | Helped |
| Add the diversity term (instead of α=1) (ablation) | RDSS with FreeMatch · CIFAR-100, 10000 labels | Top-1 accuracy: 73.05$\pm$0.18 -> 73.13$\pm$0.19 | No clear effect |
| Vary α among 0.40, 0.80, 0.90, 0.95, and 0.98 instead of using α=1-1/√m (ablation) | RDSS with FreeMatch · CIFAR-10 and CIFAR-100 | Top-1 accuracy | Mixed |
| Use RDSS instead of random sampling | FlexMatch and FreeMatch · CIFAR-10 and CIFAR-100; SVHN; STL-10 | Top-1 accuracy | Helped |
Surprising findings
- Most SSAL approaches (CoreSetSSL, MMA, CBSSAL, TOD-Semi) fall below random sampling under extremely low budgets, attributed to dependence on a weak model.
- Stratified sampling, which uses true class labels, is not a practical baseline but is reported as an upper reference.
Limitations
- The theoretical assumptions (RKHS assumptions, and the minimum-below-mean assumption for GKHR) are strong, and the authors note the latter may be unrealistic when m is not small.
- Experiments only cover image classification with CLIP features and two SSL frameworks, and no other modalities are tested.
- The optimal alpha interval is set heuristically as 1-1/sqrt(m) and is derived from order-of-magnitude arguments.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 SVHN STL-10 ImageNet-1k
Benchmarks: CIFAR-10 CIFAR-100 SVHN STL-10 ImageNet-1k
Metrics: top-1 accuracy mean and standard deviation over five runs
Baselines: Stratified Random k-Means USL ActiveFT CoreSet VAAL LearnLoss MCDAL CoreSetSSL MMA CBSSAL
Builds on: FlexMatch FreeMatch kernel herding Maximum Mean Discrepancy core-set approach
Closest papers by meaning
- CGMatch: A Different Perspective of Semi-supervised Learning (CVPR 2025 · similarity 0.90)
- Towards Cost-Effective Learning: A Synergy of Semi-Supervised and Active Learning (CVPR 2025 · similarity 0.89)
- Towards Realistic Model Selection for Semi-supervised Learning (ICML 2024 · similarity 0.88)
- Propagation Regularizer for Semi-Supervised Learning With Extremely Scarce Labeled Samples (CVPR 2022 · similarity 0.88)
- Erasing the Bias: Fine-Tuning Foundation Models for Semi-Supervised Learning (ICML 2024 · similarity 0.87)
- Enhanced Sample Selection with Confidence Tracking: Identifying Correctly Labeled Yet Hard-to-Learn Samples in Noisy Data (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.