Research map / Label-efficient and robust learning / Semi-supervised learning
Leveraging Local Variance for Pseudo-Label Selection in Semi-supervised Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Local Variance Match (LVM), which filters pseudo-labels in semi-supervised learning by requiring both high posterior probability and low local variance under input noise, and reports SOTA error rates on CIFAR-10/100, SVHN, ImageNet and an AISHELL speech setting.
Problem
Pseudo-label selection in semi-supervised learning relies only on posterior probability magnitude, which fails when sparsely labeled models are confidently wrong.
Approach
Adds local variance of the model's predicted probability for the predicted class (estimated by Gaussian input perturbations) as a second selection criterion alongside a confidence threshold, so pseudo-labels in unstable regions are discarded.
Key results
- LVM reaches 5.41% error on CIFAR-10 with 10 labels versus 8.07% for FreeMatch, close to the 4.62% fully supervised reference.
- On CIFAR-100 with 400 labels LVM achieves 35.87% error, versus 37.98% for FreeMatch.
- On ImageNet with 100k labels LVM reports 39.31% top-1 and 17.12% top-5 error, improving on FreeMatch's 40.57% and 18.77%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add local-variance-based pseudo-label selection | LVM · CIFAR-10, 10 labels | error rate (%): 8.07 -> 5.41 | Helped |
| Add local-variance-based pseudo-label selection | LVM · CIFAR-10, 40 labels | error rate (%): 4.90 -> 4.87 | Helped |
| Add local-variance-based pseudo-label selection | LVM · CIFAR-10, 250 labels | error rate (%): 4.88 -> 4.84 | Helped |
| Add local-variance-based pseudo-label selection | LVM · CIFAR-10, 4000 labels | error rate (%): 4.10 -> 4.19 | Hurt |
| Add local-variance-based pseudo-label selection | LVM · CIFAR-100, 400 labels | error rate (%): 37.98 -> 35.87 | Helped |
| Add local-variance-based pseudo-label selection | LVM · CIFAR-100, 2500 labels | error rate (%): 26.03 -> 26.52 | Hurt |
| Add local-variance-based pseudo-label selection | LVM · SVHN, 40 labels | error rate (%): 1.97 -> 1.94 | Helped |
| Add local-variance-based pseudo-label selection | LVM · SVHN, 250 labels | error rate (%): 1.92 -> 1.94 | Hurt |
| Add local-variance-based pseudo-label selection | LVM · ImageNet, 100k labeled data | top-1 error: 40.57 -> 39.31 | Helped |
| Add local-variance-based pseudo-label selection | LVM · ImageNet, 100k labeled data | top-5 error: 18.77 -> 17.12 | Helped |
| Add local-variance filtering at threshold 0.05 with posterior threshold 0.025 (ablation) | Speech recognition LVM · AISHELL-1 test with AISHELL-2 pseudo-labels | Word Error Rate (%): 4.78 -> 4.71 | Helped |
| Use local variance threshold 0.1 instead of no local variance filtering (ablation) | Speech recognition LVM, posterior threshold 0.03 · AISHELL-1 test with AISHELL-2 pseudo-labels | Word Error Rate (%): 4.72 -> 4.74 | Hurt |
Surprising findings
- On SVHN with 250 labels, LVM (26.52% error) is slightly worse than FreeMatch-level baselines (26.47%) and on CIFAR-100 with 2500 labels is not best, so gains are not uniform across settings.
Limitations
- Theoretical analysis is restricted to a simplified 1D two-Gaussian binary case and the authors note that characterizing LVM's theoretical properties remains future work.
- Local variance requires extra forward passes on perturbed inputs, adding computational cost.
- Speech experiments use an offline pseudo-labeling strategy rather than the full online method.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 SVHN ImageNet AISHELL-1 AISHELL-2
Benchmarks: CIFAR-10 CIFAR-100 SVHN ImageNet AISHELL-1 test set
Metrics: error rate (%) top-1 error top-5 error word error rate (%) mask ratio pseudo-label accuracy
Baselines: Pseudo Label MeanTeacher MixMatch ReMixMatch UDA Dash FlexMatch FreeMatch FixMatch USB
Builds on: FixMatch FreeMatch FlexMatch UDA
Closest papers by meaning
- Bi-Level Optimization for Semi-Supervised Learning with Pseudo-Labeling (AAAI 2025 · similarity 0.88)
- Int*-Match: Balancing Intra-Class Compactness and Inter-Class Discrepancy for Semi-Supervised Speaker Recognition (AAAI 2025 · similarity 0.88)
- Keep It on a Leash: Controllable Pseudo-label Generation Towards Realistic Long-Tailed Semi-Supervised Learning (NeurIPS 2025 · similarity 0.88)
- CGMatch: A Different Perspective of Semi-supervised Learning (CVPR 2025 · similarity 0.87)
- Rethinking Confidence Scores and Thresholds in Pseudolabeling-based SSL (ICML 2025 · similarity 0.87)
- Semi-Supervised Regression with Heteroscedastic Pseudo-Labels (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.