Research map / Label-efficient and robust learning / Semi-supervised learning
Rethinking Confidence Scores and Thresholds in Pseudolabeling-based SSL
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PabLO, a framework that learns confidence scores and thresholds for pseudolabeling by maximizing pseudolabel coverage under an explicit error tolerance, and plugs it into FixMatch/FreeMatch-style SSL methods.
Problem
Pseudolabeling-based semi-supervised learning relies on overconfident, miscalibrated confidence scores and heuristic thresholds, with no explicit control over pseudolabel error, hurting test accuracy and training efficiency.
Approach
Formulates pseudolabeling as a constrained optimization that maximizes coverage subject to a pseudolabel error bound, learns a 2-layer neural confidence function via differentiable sigmoid surrogates, and estimates per-class or joint thresholds on held-out validation data with a statistical error bound.
Key results
- Fixmatch + Ours reaches 89.72% on CIFAR-10 with 40 labels vs 57.16% for Fixmatch, and 67.44% vs 30.38% on CIFAR-100 with 400 labels.
- On CIFAR-100 with 2500 labels, Fixmatch + Ours achieves 69.10% vs 59.09% for Fixmatch; Freematch + Ours achieves 68.76% vs 63.13%.
- Lower error tolerance epsilon generally yields better accuracy on CIFAR-10/100 (e.g., Fixmatch + Ours 93.05% at epsilon=0.01 vs 56.72% at epsilon=0.4).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Integrate PabLO into Fixmatch | Fixmatch · CIFAR-100, 400 labels | Top-1 accuracy: 43.88 ± 3.60 -> 67.44 ± 1.14 | Helped |
| Integrate PabLO into Fixmatch | Fixmatch · CIFAR-100, 2500 labels | Top-1 accuracy: 59.09 ± 1.10 -> 69.10 ± 0.45 | Helped |
| Integrate PabLO into Freematch | Freematch · CIFAR-100, 2500 labels | Top-1 accuracy: 63.13 ± 0.46 -> 68.76 ± 1.38 | Helped |
| Integrate PabLO into Fixmatch | Fixmatch · SVHN, 250 labels | Top-1 accuracy: 97.57 ± 0.08 -> 96.54 ± 0.13 | Hurt |
| Set pseudolabeling error tolerance to 0.01 instead of 0.4 (ablation) | Fixmatch + Ours · CIFAR-10 | Top-1 accuracy: 56.72 ± 22.25 -> 93.05 ± 0.54 | Helped |
| Set pseudolabeling error tolerance to 0.01 instead of 0.4 (ablation) | Freematch + Ours · CIFAR-10 | Top-1 accuracy: 52.17 ± 25.36 -> 92.11 ± 1.18 | Helped |
| Set pseudolabeling error tolerance to 0.01 instead of 0.4 (ablation) | Fixmatch + Ours · CIFAR-100 | Top-1 accuracy: 23.58 ± 18.21 -> 69.19 ± 1.13 | Helped |
| Set pseudolabeling error tolerance to 0.01 instead of 0.4 (ablation) | Freematch + Ours · CIFAR-100 | Top-1 accuracy: 24.09 ± 17.22 -> 70.13 ± 0.67 | Helped |
| Use a decreasing rather than increasing error-tolerance schedule (ablation) | Fixmatch + Ours · CIFAR-10 | Top-1 accuracy: 35.38 ± 29.27 -> 91.11 ± 1.31 | Helped |
| Use a decreasing rather than increasing error-tolerance schedule (ablation) | Fixmatch + Ours · CIFAR-100 | Top-1 accuracy: 19.56 ± 3.07 -> 65.78 ± 1.36 | Helped |
| Increase calibration and threshold-estimation data from 250 to 500 points each (ablation) | Fixmatch + Ours · CIFAR-10 | Top-1 accuracy: 82.67 ± 7.08 -> 91.74 ± 0.41 | Helped |
| Increase calibration and threshold-estimation data from 250 to 500 points each (ablation) | Freematch + Ours · CIFAR-10 | Top-1 accuracy: 82.13 ± 7.93 -> 92.33 ± 0.49 | Helped |
Surprising findings
- Pseudolabel accumulation hurts baseline methods but has little effect on the proposed method, which already maximizes coverage at high quality.
- Scheduling error tolerance from small to large severely degrades accuracy, while the reverse schedule works well.
Limitations
- Requires a portion of validation labeled data (calibration and threshold sets) which may be costly in practice.
- Higher per-iteration compute cost than baseline SSL methods.
- Pseudolabel accumulation is only warranted when pseudolabel quality is high.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 SVHN
Benchmarks: CIFAR-10 CIFAR-100 SVHN
Metrics: top-1 accuracy pseudolabeling coverage pseudolabeling error
Baselines: FixMatch FreeMatch SoftMatch AdaMatch FixMatch + MR FixMatch + BaM FreeMatch + MR FreeMatch + BaM
Builds on: FixMatch FreeMatch Vishwakarma et al. 2024 auto-labeling confidence framework Bayesian Model Averaging for SSL (BaM) Margin regularization for SSL
Closest papers by meaning
- Bi-Level Optimization for Semi-Supervised Learning with Pseudo-Labeling (AAAI 2025 · similarity 0.89)
- CGMatch: A Different Perspective of Semi-supervised Learning (CVPR 2025 · similarity 0.89)
- Keep It on a Leash: Controllable Pseudo-label Generation Towards Realistic Long-Tailed Semi-Supervised Learning (NeurIPS 2025 · similarity 0.88)
- Semi-Supervised Regression with Heteroscedastic Pseudo-Labels (NeurIPS 2025 · similarity 0.88)
- Contrastive Credibility Propagation for Reliable Semi-supervised Learning (AAAI 2024 · similarity 0.88)
- SemiReward: A General Reward Model for Semi-supervised Learning (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.