Research map / Label-efficient and robust learning / Semi-supervised learning
A Square Peg in a Square Hole: Meta-Expert for Long-Tailed Semi-Supervised Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Meta-Expert, an LTSSL method that dynamically assigns head/medium/tail experts per sample via a learned DEA module and fuses multi-depth features (MFF) to reduce head-class bias under distribution mismatch.
Problem
Long-tailed semi-supervised learning where labeled class distribution is long-tailed and unlabeled distribution differs (consistent, uniform, inverse), causing biased and noisy pseudo-labels.
Approach
Instead of using all experts equally, a dynamic expert assignment module estimates each sample's class membership (head/medium/tail) from encoder features and expert logits, softly weighting three logit-adjusted experts for pseudo-labeling and prediction; a multi-depth feature fusion combines shallow (less biased) and deep (more discriminative) features.
Key results
- On CIFAR-10-LT with (N1,M1,gamma)=(500,4000,200), Meta-Expert surpasses all previous baselines by up to 2.47 pp in the consistent case.
- On CIFAR-10-LT with (N1,M1,gamma_l,gamma_u)=(1500,3000,200,1/200), Meta-Expert surpasses the previous SOTA by 1.64 pp.
- Ablation on CIFAR-10-LT (gamma=200): DEA alone +0.65 pp, MFF alone +1.26 pp, combined +3.10 pp over the base.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Meta-Expert instead of the strongest listed baseline | LTSSL with FixMatch base · CIFAR-10-LT across consistent and mismatched distributions | accuracy: 74.39 ± 0.46 vs 67.45 ± 1.27; 85.71 ± 0.03 vs 84.07 ± 0.40 | Helped |
| Use Meta-Expert instead of the strongest listed baseline | LTSSL with FixMatch base · STL-10-LT | accuracy: 71.19 ± 0.07 vs 69.24 ± 0.95; 79.98 ± 0.33 vs 78.71 ± 0.24 | Helped |
| Use Meta-Expert instead of the strongest listed baseline | LTSSL with FixMatch base · SVHN-LT | accuracy: 93.56 ± 0.09 vs 93.30 ± 0.15; 94.24 ± 0.19 vs 94.76 ± 0.41 | Mixed |
| Add DEA expert assignment (ablation) | Meta-Expert · CIFAR-10-LT, γu=200 | accuracy: 78.57 -> 79.22 | Helped |
| Add MFF multi-depth feature fusion (ablation) | Meta-Expert · CIFAR-10-LT, γu=200 | accuracy: 78.57 -> 79.83 | Helped |
| Add DEA and MFF together (ablation) | Meta-Expert · CIFAR-10-LT, γu=200 | accuracy: 78.57 -> 81.67 | Helped |
| Add MFF without DEA (ablation) | Meta-Expert · CIFAR-10-LT, γu=1/200 | accuracy: 84.40 -> 83.10 | Hurt |
| Use addition rather than concatenation in MFF (ablation) | Meta-Expert · CIFAR-10-LT, γu=1 | accuracy: 83.38 -> 83.96 | Helped |
| Assign experts using DEA rather than CPE's expert strategy | CIFAR-10-LT, consistent distribution · CIFAR-10-LT, γl=200, γu=200 | pseudo-labeling error rate: 30.07 -> 21.17 | Helped |
| Assign experts using DEA rather than CPE's expert strategy | CIFAR-10-LT, consistent distribution · CIFAR-10-LT, γl=200, γu=200 | unlabeled data utilization ratio: 84.86 -> 95.31 | Helped |
Surprising findings
- MFF slightly hurts in the inverse case (gamma_u=1/200) when used without DEA, which the authors attribute to using all experts for pseudo-labels.
- Shallow features are less discriminative but more balanced across head/tail classes, while deep features are more discriminative but head-biased.
Limitations
- Expert assignment depends on a learned DEA module; using ground-truth membership (Upper-E) performs better, so assignment accuracy is a bottleneck.
- Evaluated only on three small image benchmarks with WRN-28-2 in the USB codebase.
- Theoretical bound relies on a pseudo-labeling error term that is not directly estimated in practice.
Experimental setup
Datasets: CIFAR-10-LT SVHN-LT STL-10-LT
Benchmarks: CIFAR-10-LT SVHN-LT STL-10-LT
Metrics: accuracy pseudo-label error rate F1 score
Baselines: FixMatch SAW Adsh DePL ACR BaCon CPE SimPro
Builds on: FixMatch CPE ACR USB
Closest papers by meaning
- Three Heads Are Better than One: Complementary Experts for Long-Tailed Semi-supervised Learning (AAAI 2024 · similarity 0.93)
- Keep It on a Leash: Controllable Pseudo-label Generation Towards Realistic Long-Tailed Semi-Supervised Learning (NeurIPS 2025 · similarity 0.88)
- Breaking Long-Tailed Learning Bottlenecks: A Controllable Paradigm with Hypernetwork-Generated Diverse Experts (NeurIPS 2024 · similarity 0.87)
- Continuous Contrastive Learning for Long-Tailed Semi-Supervised Recognition (NeurIPS 2024 · similarity 0.87)
- SimPro: A Simple Probabilistic Framework Towards Realistic Long-Tailed Semi-Supervised Learning (ICML 2024 · similarity 0.87)
- Long-Tailed Partial Label Learning by Head Classifier and Tail Classifier Cooperation (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.