Research map / Label-efficient and robust learning / Semi-supervised learning
Self-training theory
Analyzes self-training, contrastive learning and unlabeled data with theoretical guarantees.
86 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Semi-supervised learning, keyword cluster “semi · examples · classification”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
22 papers in 2023–24 and 13 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 13.8% to 8.2%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Modeling Evolving Relationships Between Characters in Literary Novels (AAAI 2016 · 45 citations)
- 2017 · Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results (NeurIPS 2017 · 1,557 citations)
- 2018 · DeepHeart: Semi-Supervised Sequence Learning for Cardiovascular Risk Prediction (AAAI 2018 · 142 citations)
- 2019 · Infinite Mixture Prototypes for Few-shot Learning (ICML 2019 · 79 citations)
- 2020 · Towards Discriminability and Diversity: Batch Nuclear-Norm Maximization Under Label Insufficient Situations (CVPR 2020 · 418 citations)
- 2021 · Exponential Moving Average Normalization for Self-Supervised and Semi-Supervised Learning (CVPR 2021 · 117 citations)
- 2022 · Towards Semi-Supervised Deep Facial Expression Recognition With an Adaptive Confidence Margin (CVPR 2022 · 105 citations)
- 2023 · Harnessing Hard Mixed Samples with Decoupled Regularizer (NeurIPS 2023 · 13 citations)
- 2024 · Multilevel Attention Network with Semi-supervised Domain Adaptation for Drug-Target Prediction (AAAI 2024 · 12 citations)
- 2025 · Semi-IIN: Semi-Supervised Intra-Inter Modal Interaction Learning Network for Multimodal Sentiment Analysis (AAAI 2025 · 9 citations)
- 2026 · Leveraging Lineage Barcodes as Natural Augmentations for Contrastive Learning of Cell Fate in scRNA-seq Data (ICML 2026 · 1 citation)
Earliest papers here
- Modeling Evolving Relationships Between Characters in Literary Novels (AAAI 2016 · 45 citations)
- Global Distant Supervision for Relation Extraction (AAAI 2016 · 43 citations)
- Acquiring Knowledge of Affective Events from Blogs Using Label Propagation (AAAI 2016 · 29 citations)
What has been tried
79 papers here report 887 measured changes (a component added, removed or swapped, compared with a baseline): helped 74%, no clear effect 9%, hurt 15%, mixed 3%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Add BERT embeddings instead of Word2Vec and GCN embeddings | BindingDB cross-domain · AUC | 0.573 -> 0.687 | Helped | Multilevel Attention Network with Semi-supervised Domain Adaptation for Drug-Target Prediction (AAAI 2024 · 12 citations) |
| add consistency regularization using unlabeled data | Youtube · AVE MAE (%) | 65.04 -> 64.21 | Helped | Data Augmented Graph Neural Networks for Personality Detection (AAAI 2024 · 10 citations) |
| Add masked attention without semi-supervised training | MOSEI · MAE | 0.509 -> 0.499 | Helped | Semi-IIN: Semi-Supervised Intra-Inter Modal Interaction Learning Network for Multimodal Sentiment Analysis (AAAI 2025 · 9 citations) |
| add domain classifier | ADC2004 · OA | 73.1 -> 76.7 | Helped | MCSSME: Multi-Task Contrastive Learning for Semi-supervised Singing Melody Extraction from Polyphonic Music (AAAI 2024 · 9 citations) |
| Add Dual-Ele to the baseline without the three proposed components | Gym99, Gym288, FineDiving · top-1 accuracy | 32.6 -> 34.8; 22.7 -> 25.4; 60.4 -> 64.6 | Helped | SeFAR: Semi-supervised Fine-grained Action Recognition with Temporal Perturbation and Learning Stabilization (AAAI 2025 · 7 citations) |
| train on RA-labeled pseudo instances rather than QP-labeled pseudo instances | KdConv test · Unigram F1 | 58.43 -> 65.60 | Helped | Response Enhanced Semi-supervised Dialogue Query Generation (AAAI 2024 · 4 citations) |
| Disable student feedback to the teacher | LK · Local Error | 12.84 -> 23.60 | Hurt | Semi-supervised Learning of Dynamical Systems with Neural Ordinary Differential Equations: A Teacher-Student Model Approach (AAAI 2024 · 2 citations) |
| Add intra-trend ranking awareness to separate spatial attention layers | BP4D · MAE | .84 -> .78 | Helped | Trend-Aware Supervision: On Learning Invariance for Semi-supervised Facial Action Unit Intensity Estimation (AAAI 2024 · 1 citation) |
| Use model prediction only for sample-gain estimation | ImageNet-1K, 10% labels · accuracy | 79.2 -> 78.5 | Hurt | Info-Coevolution: An Efficient Framework for Data Model Coevolution (ICML 2025 · 0 citations) |
| Use ACPs that are only a function of X | Semi-supervised estimation under covariate shift · estimation efficiency | No clear effect | Towards the Efficient Inference by Incorporating Automated Computational Phenotypes under Covariate Shift (ICML 2025 · 0 citations) | |
| Add DAREM and semi-supervised contrastive learning rather than using an encoder-only Transformer | 19 selected datasets · Avg. Acc. | 0.824 -> 0.841; 0.826 -> 0.841 | Helped | STaRFormer: Semi-Supervised Task-Informed Representation Learning via Dynamic Attention-Based Regional Masking for Sequential Data (NeurIPS 2025 · 0 citations) |
| Add fine-tuning sensitivity analysis to Cross-Model Trigger Reverse | CIFAR10, Blended, 4×4 trigger, ResNet-18 · FPR | 10.0 -> 0.0 | Helped | Lie Detector: Unified Backdoor Detection via Cross-Examination Framework (NeurIPS 2025 · 0 citations) |
What reviewers will expect
Datasets and baselines named in the 8 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- CIFAR-10 · 2 of 8 papers
- COCO · 2 of 8 papers
Baselines
- FixMatch · 3 of 8 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 35 papers; from OpenAlex and the papers' affiliations).
- Xiamen University · 2 papers
- Carnegie Mellon University · 2 papers
- National University of Singapore · 2 papers
Use it
Download BibTeX of 40 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Holistic semi-supervised pipelines (125 papers)
- Safe and realistic semi-supervised learning (70 papers)
- GAN-based semi-supervised learning (61 papers)
Elsewhere in the topic
- Confidence-based pseudo-label selection (134 papers)
- Semi-supervised medical segmentation (50 papers)
- Consistency training for semi-supervised segmentation (49 papers)
- Pseudo-labeling for specialized tasks (48 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).