Research map / Label-efficient and robust learning / Semi-supervised learning
SimPro: A Simple Probabilistic Framework Towards Realistic Long-Tailed Semi-Supervised Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SimPro, an EM-based long-tailed semi-supervised framework that decouples conditional and marginal class distributions, estimates the unlabeled class prior in closed form, and uses the resulting Bayes classifier for pseudo-labels, without assuming the unlabeled distribution.
Problem
Semi-supervised learning where labeled data is long-tailed and the unlabeled class distribution is unknown and possibly mismatched with the labeled distribution (ReaLTSSL).
Approach
Decouple the conditional P(x|y) (network parameters trained by SGD) from the marginal class prior P(y) (closed-form estimate from pseudo-label averages, updated via moving average), so the prior enters a Bayes classifier used to generate pseudo-labels in the E-step; no predefined anchor distributions are required.
Key results
- On CIFAR10-LT (gamma_l=150) with unlabeled head-tail distribution, SimPro reaches 81.0% vs 79.0% for ACR; with middle distribution 82.6% vs 77.9% for ACR.
- On CIFAR10-LT with uniform unlabeled distribution (gamma_u=1), SimPro gets 93.6% vs ACR 91.9%.
- On ImageNet-1k (gamma_t=1, 32x32), SimPro gets 19.7% vs ACR 13.8%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SimPro instead of ACR† | FixMatch on CIFAR10-LT, γl=150 · uniform unlabeled distribution | top-1 accuracy (%): 91.9 ± 0.02 -> 93.6 ± 0.08 | Helped |
| Use SimPro instead of ACR† | FixMatch on CIFAR10-LT, γl=150 · middle unlabeled distribution | top-1 accuracy (%): 73.8 ± 0.83 -> 82.6 ± 0.38 | Helped |
| Add M-step marginal-distribution estimation (ablation) | SimPro on CIFAR10-LT, γl=150 · uniform unlabeled distribution | top-1 accuracy (%): 35.3 -> 92.6 | Helped |
| Add E-step use of the estimated distribution to build the Bayes classifier (ablation) | SimPro on CIFAR10-LT, γl=150 · middle unlabeled distribution | top-1 accuracy (%): 64.9 -> 82.6 | Helped |
| Set μ=M/N instead of the smaller default (ablation) | SimPro on CIFAR10-LT, γl=150 · consistent unlabeled distribution | top-1 accuracy (%): 52.7 -> 75.2 | Helped |
| Set μ=M/N instead of the smaller default (ablation) | FixMatch on CIFAR10-LT, γl=150 · consistent unlabeled distribution | top-1 accuracy (%): 62.9 -> 40.7 | Hurt |
| Set μ=M/N instead of the smaller default (ablation) | ACR on CIFAR10-LT, γl=150 · reversed unlabeled distribution | top-1 accuracy (%): 83.2 -> 58.9 | Hurt |
| Add the loss-balancing factor α (ablation) | SimPro on CIFAR10-LT · uniform unlabeled distribution, γl=150 | top-1 accuracy (%): 92.1 -> 93.6 | Helped |
| Add the loss-balancing factor α (ablation) | SimPro on STL10-LT · γl=20 | top-1 accuracy (%): 76.4 -> 83.0 | Helped |
| Use predefined anchor distributions in SimPro (SimPro★) (ablation) | SimPro on CIFAR10-LT, γl=150 · consistent unlabeled distribution | top-1 accuracy (%): 74.2 -> 80.0 | Helped |
| Use predefined anchor distributions in SimPro (SimPro★) (ablation) | SimPro on CIFAR10-LT, γl=150 · middle unlabeled distribution | top-1 accuracy (%) | No clear effect |
Surprising findings
- Using the anchor distributions in SimPro (SimPro*) improves results further, but the anchor-free SimPro underperforms ACR on the consistent unlabeled distribution (74.2% vs 77.0% on CIFAR10-LT).
- Standard FixMatch and ACR degrade sharply when mu=M/N=8 is used, while SimPro is robust to this larger unlabeled ratio.
Limitations
- Underperforms ACR on the consistent unlabeled distribution without anchors
- Requires a confidence threshold that is tuned per dataset (0.95 on CIFAR10, 0.2 on CIFAR100)
- Regret bound is derived only for binary classification
Experimental setup
Datasets: CIFAR10-LT CIFAR100-LT STL10-LT ImageNet-127 ImageNet-1k
Benchmarks: CIFAR10-LT CIFAR100-LT STL10-LT ImageNet-127 ImageNet-1k
Metrics: top-1 accuracy
Baselines: FixMatch CReST+ DASO ACR CPE CoSSL DARP
Builds on: FixMatch ACR EM algorithm logit adjustment
Closest papers by meaning
- Keep It on a Leash: Controllable Pseudo-label Generation Towards Realistic Long-Tailed Semi-Supervised Learning (NeurIPS 2025 · similarity 0.88)
- Continuous Contrastive Learning for Long-Tailed Semi-Supervised Recognition (NeurIPS 2024 · similarity 0.87)
- CGMatch: A Different Perspective of Semi-supervised Learning (CVPR 2025 · similarity 0.87)
- A Square Peg in a Square Hole: Meta-Expert for Long-Tailed Semi-Supervised Learning (ICML 2025 · similarity 0.87)
- Three Heads Are Better than One: Complementary Experts for Long-Tailed Semi-supervised Learning (AAAI 2024 · similarity 0.87)
- BEM: Balanced and Entropy-based Mix for Long-Tailed Semi-Supervised Learning (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.