Research map / Label-efficient and robust learning / Semi-supervised learning
Dual Student: Breaking the Limits of the Teacher in Semi-Supervised Learning
Zhanghan Ke, Daoye Wang, Qiong Yan, Jimmy Ren, Rynson W.H. Lau
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace EMA teacher with a second independent student plus stabilization constraint | Mean Teacher · CIFAR-10 (1k labels) | test error rate (%): 18.78 -> 15.74 | Improved |
| add stabilization constraint instead of plain consistency between students (ablation) | Dual Student · CIFAR-10 (1k labels) | test error rate (%): 17.38 -> 15.74 | Improved |
| use a confidence threshold xi=0.8 for stable samples (ablation) | Dual Student · CIFAR-10 (1k labels) | test error rate (%): 16.49 -> 15.74 | Improved |
| use a confidence threshold xi=0.4 for stable samples (ablation) | Dual Student · CIFAR-100 (10k labels) | test error rate (%): 33.67 -> 33.08 | Improved |
| train four students (Multiple Student) instead of two | Dual Student · CIFAR-10 (1k labels) | test error rate (%): 15.74 -> 14.97 | Improved |
| train eight students instead of four | Multiple Student · CIFAR-10 (1k labels) | test error rate (%): 14.97 -> 14.77 | No clear effect |
| replace one student with a larger ResNet (Imbalanced Student) | Dual Student · CIFAR-10 (1k labels) | test error rate (%): 15.74 -> 12.39 | Improved |
| use the small 3.53M-param ResNet as stronger student | Dual Student · CIFAR-10 (1k labels) | test error rate (%): 15.74 -> 13.43 | Improved |
| apply Dual Student instead of Mean Teacher | Mean Teacher · ImageNet (10% labels, top-1) | test error rate (%): 37.83 -> 36.48 | Improved |
| apply Dual Student instead of Mean Teacher | Mean Teacher · USPS to MNIST domain adaptation | top-1 accuracy (%): 80.41 -> 91.50 | Improved |
| use Dual Student instead of MT+FSWA with all labels | MT + FSWA · CIFAR-10 (all labels) | test error rate (%): 5.14 -> 5.20 | Hurt |
| train Dual Student for 600 epochs instead of 300 | Dual Student · CIFAR-10 (1k labels) | test error rate (%): 15.74 -> 14.17 | Improved |
Later papers that compared with this method
2 later accepted papers report DualStudent as a baseline in their results tables; 2 of them report a better number on most of the results they share with it.
- In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learning (ICLR 2021 · better on 1 of 1 shared result)
- Density-Aware Graph for Deep Semi-Supervised Visual Recognition (CVPR 2020 · better on 1 of 1 shared result)
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Switching Temporary Teachers for Semi-Supervised Semantic Segmentation (NeurIPS 2023 · similarity 0.89)
- Newton-coupled Dual-Teacher Semi-supervised Learning Framework (ICML 2026 · similarity 0.89)
- Time-Consistent Self-Supervision for Semi-Supervised Learning (ICML 2020 · similarity 0.88)
- MaxMatch: Semi-Supervised Learning With Worst-Case Consistency (TPAMI 2022 · similarity 0.88)
- ConMatch: Semi-Supervised Learning with Confidence-Guided Consistency Regularization (ECCV 2022 · similarity 0.88)
- Erasing the Bias: Fine-Tuning Foundation Models for Semi-Supervised Learning (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
