Research map / Label-efficient and robust learning / Semi-supervised learning
Semi-supervised Vision Transformers at Scale
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace FixMatch with EMA-Teacher (ablation) | ViT-Base with MAE self-pretraining · ImageNet 10% labels | top-1 accuracy: 74.8 -> 78.1 | Helped |
| Replace FixMatch with EMA-Teacher (ablation) | ViT without self-pretraining · ImageNet 10% labels | training convergence | Helped |
| Add pseudo mixup on unlabeled data (ablation) | EMA-Teacher, ViT-Small without pretraining · ImageNet 10% labels | top-1 accuracy: 65.6 -> 68.3 | Helped |
| Restrict pseudo mixup to confident samples (ablation) | Pseudo Mixup · ImageNet SSL | top-1 accuracy: about 0.5 points | Helped |
| Weight mixed samples according to pseudo-label confidence (ablation) | Pseudo Mixup+ · ImageNet SSL | top-1 accuracy: 1-2 points | Helped |
| Add probabilistic pseudo mixup (ablation) | EMA-Teacher, ViT-Base with MAE · ImageNet 1% labels | top-1 accuracy: 65.3 -> 71.0 | Helped |
| Use MAE self-pretraining (ablation) | ViT-Base with EMA-Teacher · ImageNet 10% labels | top-1 accuracy: +6.2 points | Helped |
| Use MAE self-pretraining (ablation) | ViT-Base with probabilistic pseudo mixup · ImageNet 10% labels | top-1 accuracy: +9.2 points | Helped |
| Use DINO rather than MAE self-pretraining (ablation) | ViT-Base with Semi-ViT · ImageNet 1% labels | top-1 accuracy: +2.1 points | Helped |
| Use DINO rather than MAE self-pretraining (ablation) | ViT-Base with Semi-ViT · ImageNet 10% labels | top-1 accuracy: +0.5 points | Helped |
| Apply probabilistic pseudo mixup (ablation) | ConvNeXt-T with EMA-Teacher · ImageNet 10% labels | top-1 accuracy: 70.4 -> 74.1 | Helped |
Closest papers by meaning
- SemiCVT: Semi-Supervised Convolutional Vision Transformer for Semantic Segmentation (CVPR 2023 · similarity 0.91)
- Revisiting Semi-Supervised Learning in the Era of Foundation Models (NeurIPS 2025 · similarity 0.89)
- Vision Transformers Need Registers (ICLR 2024 · similarity 0.86)
- Slicing Vision Transformer for Flexible Inference (NeurIPS 2024 · similarity 0.86)
- DA-VPT: Semantic-Guided Visual Prompt Tuning for Vision Transformers (CVPR 2025 · similarity 0.85)
- Strong Baselines for Parameter-Efficient Few-Shot Fine-Tuning (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.