Research map / Label-efficient and robust learning / Semi-supervised learning
Revisiting Semi-Supervised Learning in the Era of Foundation Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
The paper benchmarks SSL methods with vision foundation models on six VTAB-derived tasks and proposes V-PET, a self-training method that ensembles pseudo-labels from multiple PEFT methods and VFM backbones via a Mean Label strategy.
Problem
Semi-supervised learning methods were designed to train networks from scratch, and it is unclear whether they remain useful when starting from vision foundation models, where frozen VFMs already perform well on standard SSL datasets but underperform on harder specialized tasks.
Approach
Labeled-only PEFT on VFMs matches or beats classic SSL methods, so PEFT-generated pseudo-labels are used for self-training with no confidence threshold, ensembled across diverse PEFT methods and VFM backbones using a Mean Label (averaged one-hot) strategy.
Key results
- V-PET with CLIP and LoRA reaches 60.5% average accuracy across 12 settings versus 54.7% for ST and 55.7% for labeled-only LoRA; with DINOv2 and AdaptFormer, V-PET reaches 61.5%.
- Labeled-only full fine-tuning matches or beats FixMatch, FlexMatch, SoftMatch and FineSSL in the fair-tuning comparison (e.g., CLIP full fine-tuning labeled-only 47.44% average vs. SoftMatch 46.34%).
- The unsupervised multi-criterion hyperparameter selection gives 2.6% mean absolute error from oracle accuracy versus 8.9% for random selection.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Ensemble pseudo-labels across PEFT methods and VFM backbones instead of using self-training with one model (ablation) | V-PET · 12 settings (average) | Top-1 accuracy: 54.7 -> 60.5; 56.7 -> 61.0; 52.9 -> 61.2; 52.5 -> 61.5 | Helped |
| Add pseudo-label ensembling across PEFT methods on one VFM (ablation) | Self-training · 12 settings (average) | Top-1 accuracy: 54.7 -> 59.3; 56.7 -> 59.7; 52.9 -> 56.7; 52.5 -> 58.0 | Helped |
| Use PEFT rather than full fine-tuning | VFM-based SSL · 12 benchmark settings | Top-1 accuracy | Helped |
| Use labeled-only PEFT rather than existing SSL methods | VFM fine-tuning · 12 benchmark settings | Top-1 accuracy | Mixed |
| Use Mean Labels instead of Mean Logits or Mean Probabilities for ensembling (ablation) | V-PET pseudo-label ensemble · Different ensemble sources and ensemble sizes | Top-1 accuracy | Helped |
| Add pseudo-label sources from different PEFT methods (ablation) | V-PET · Ensemble scaling experiments | Top-1 accuracy | Helped |
| Add pseudo-label sources from different VFM backbones (ablation) | V-PET · Ensemble scaling experiments | Top-1 accuracy | Helped |
| Increase the number of ensemble sources (ablation) | V-PET · Ensemble scaling experiments | Top-1 accuracy | Mixed |
| Combine seven unsupervised criteria rather than choose hyperparameters randomly | SSL hyperparameter tuning · 12 datasets, 5 SSL baselines, 2 VFMs, 2 PEFT methods | absolute error from oracle accuracy: 8.9±8.1 -> 2.6±4.3 | Helped |
| Replace full fine-tuning with LoRA (ablation) | VFM-based SSL · 12 benchmark settings | Top-1 accuracy | Mixed |
| Replace full fine-tuning with AdaptFormer (ablation) | VFM-based SSL · 12 benchmark settings | Top-1 accuracy | Mixed |
Surprising findings
- Labeled-only PEFT often matches or surpasses classic SSL methods, so unlabeled data provides little benefit under fair hyperparameter tuning.
- FineSSL performs poorly with full fine-tuning (19.51% average with CLIP) but is competitive with PEFT, showing that full fine-tuning can damage VFM generalization.
Limitations
- Evaluation restricted to image classification with six VTAB-derived datasets and 12 shot settings.
- Ensemble gains show diminishing returns with more sources, and per-source compute scales with the number of PEFT-VFM pairs.
- Hyperparameter tuning relies on seven unsupervised criteria, with no single criterion reliable across all scenarios.
Experimental setup
Datasets: DTD SUN397 RESISC45 Diabetic Retinopathy CLEVR-Count KITTI
Benchmarks: DTD SUN397 RESISC45 Retinopathy CLEVR-C KITTI
Metrics: Top-1 accuracy average rank absolute error from oracle test accuracy
Baselines: FixMatch FlexMatch SoftMatch FineSSL Self-training (ST) Labeled-only fine-tuning PET
Builds on: FixMatch FlexMatch SoftMatch FineSSL LoRA AdaptFormer
Closest papers by meaning
- Towards Discovering the Effectiveness of Moderately Confident Samples for Semi-Supervised Learning (CVPR 2022 · similarity 0.89)
- Erasing the Bias: Fine-Tuning Foundation Models for Semi-Supervised Learning (ICML 2024 · similarity 0.89)
- CGMatch: A Different Perspective of Semi-supervised Learning (CVPR 2025 · similarity 0.88)
- InterLUDE: Interactions between Labeled and Unlabeled Data to Enhance Semi-Supervised Learning (ICML 2024 · similarity 0.87)
- Lessons and Insights from a Unifying Study of Parameter-Efficient Fine-Tuning (PEFT) in Visual Recognition (CVPR 2025 · similarity 0.87)
- Candidate Pseudolabel Learning: Enhancing Vision-Language Models by Prompt Tuning with Unlabeled Data (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.