Research map / Label-efficient and robust learning / Semi-supervised learning
Three Heads Are Better than One: Complementary Experts for Long-Tailed Semi-supervised Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ComPlementary Experts (CPE), three logit-adjusted classifier heads on a shared backbone plus Classwise Batch Normalization, to produce better pseudo-labels for long-tailed semi-supervised learning under consistent, uniform, and inverse unlabeled distributions.
Problem
Long-tailed semi-supervised learning where labeled data are imbalanced and unlabeled data follow an unknown class distribution (consistent, uniform, or inverse), causing pseudo-labels to be skewed toward head classes.
Approach
Train three expert heads with different logit-adjustment strengths (tau = 0, 2, 4) on a shared feature extractor so that one expert yields high-quality pseudo-labels for each unlabeled distribution case, and apply classwise batch normalization (three BN layers by head/medium/tail groups) to unsupervised loss to handle feature variance mismatch.
Key results
- On CIFAR-10-LT (N1=500, M1=400, gamma_l=100, gamma_u=1), CPE surpasses ACR by 1.22 pp and other baselines by over 8 pp.
- On CIFAR-10-LT inverse case (gamma_u=1/100, N1=1500), CPE reaches 87.09 vs ACR 86.29 and FixMatch 68.92.
- Ablation on CIFAR-10-LT inverse setting shows three experts alone give 86.04 and CBN alone 81.22 vs FixMatch 68.92; combined 87.28.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CPE instead of ACR | LTSSL with WRN-28-2 · CIFAR-10-LT consistent, N1=1500, M1=3000, γ=150 | top-1 accuracy: 81.81 -> 82.25 | Helped |
| Use CPE instead of ACR | LTSSL with WRN-28-2 · CIFAR-10-LT consistent, N1=500, M1=4000, γ=100 | top-1 accuracy: 81.01 -> 80.68 | Hurt |
| Use CPE instead of ACR | LTSSL with WRN-28-2 · CIFAR-100-LT consistent, γ=10 | top-1 accuracy: 59.83 -> 59.83 | No clear effect |
| Use CPE instead of ACR | LTSSL with WRN-28-2 · CIFAR-10-LT uniform, N1=500, M1=400 | top-1 accuracy: 80.10 -> 82.32 | Helped |
| Use CPE instead of ACR | LTSSL with WRN-28-2 · STL-10-LT, γl=10 | top-1 accuracy: 73.40 -> 73.07 | Hurt |
| Add three experts to FixMatch (ablation) | FixMatch on CIFAR-10-LT · CIFAR-10-LT, γu=100 | top-1 accuracy: 76.49 -> 84.02 | Helped |
| Add three experts to FixMatch (ablation) | FixMatch on CIFAR-10-LT · CIFAR-10-LT, γu=1/100 | top-1 accuracy: 68.92 -> 86.04 | Helped |
| Add CBN to FixMatch (ablation) | FixMatch on CIFAR-10-LT · CIFAR-10-LT, γu=100 | top-1 accuracy: 76.49 -> 79.00 | Helped |
| Add CBN to FixMatch (ablation) | FixMatch on CIFAR-10-LT · CIFAR-10-LT, γu=1/100 | top-1 accuracy: 68.92 -> 81.22 | Helped |
| Add CBN to the three-expert method (ablation) | Three-expert FixMatch on CIFAR-10-LT · CIFAR-10-LT, γu=100 | top-1 accuracy: 84.02 -> 84.79 | Helped |
| Add CBN to the three-expert method (ablation) | Three-expert FixMatch on CIFAR-10-LT · CIFAR-10-LT, γu=1/100 | top-1 accuracy: 86.04 -> 87.28 | Helped |
Surprising findings
- Low-precision pseudo-labels from the first (regular CE) head barely harm representation learning, since tail-class accuracy is mainly driven by tail recall.
Limitations
- Gains on CIFAR-100-LT and STL-10-LT are comparable to ACR rather than clearly superior.
- Uses three experts, increasing head count and training cost relative to single-head methods.
- Logit adjustment strengths and head/medium/tail split are fixed hyperparameters chosen for all datasets.
Experimental setup
Datasets: CIFAR-10-LT CIFAR-100-LT STL-10-LT
Benchmarks: CIFAR-10-LT CIFAR-100-LT STL-10-LT
Metrics: top-1 test accuracy F1 score of pseudo-labels
Baselines: FixMatch DARP CReST CReST+ ABC DASO CoSSL SAW Adsh DePL RDA ACR
Builds on: FixMatch logit adjustment ACR DASO USB
Closest papers by meaning
- A Square Peg in a Square Hole: Meta-Expert for Long-Tailed Semi-Supervised Learning (ICML 2025 · similarity 0.93)
- Keep It on a Leash: Controllable Pseudo-label Generation Towards Realistic Long-Tailed Semi-Supervised Learning (NeurIPS 2025 · similarity 0.89)
- Continuous Contrastive Learning for Long-Tailed Semi-Supervised Recognition (NeurIPS 2024 · similarity 0.88)
- Long-Tailed Partial Label Learning by Head Classifier and Tail Classifier Cooperation (AAAI 2024 · similarity 0.87)
- CGMatch: A Different Perspective of Semi-supervised Learning (CVPR 2025 · similarity 0.87)
- SimPro: A Simple Probabilistic Framework Towards Realistic Long-Tailed Semi-Supervised Learning (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.