Research map / Label-efficient and robust learning / Semi-supervised learning
Navigating the Pitfalls of Active Learning Evaluation: A Systematic Framework for Meaningful Performance Assessment
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Evaluate AL across imbalanced datasets and roll-out datasets | Active learning query methods · CIFAR-10 LT, ISIC-2019, MIO-TCD | performance relative to random querying | Helped |
| Use BADGE instead of random querying | Active learning · Tested datasets, budgets, and query sizes | classification performance | Helped |
| Use Entropy instead of random querying | Active learning · MIO-TCD with ST | classification performance | Hurt |
| Use BALD instead of random querying | Active learning with Self-SL · MIO-TCD | classification performance | Hurt |
| Use a smaller starting budget | Active learning query methods · Most tested datasets and methods | classification performance relative to random querying | No clear effect |
| Add lightweight hyperparameter tuning on a validation set | ST classifiers · CIFAR-10/100 | accuracy: \sim 44 \% trained on 200 samples compared to ... 1k samples | Helped |
| Add SimCLR Self-SL pretraining | Classifier training · Across experiments | classification performance | Helped |
| Add FixMatch Semi-SL training | Classifier training · CIFAR-10/100 | classification performance | Helped |
| Add FixMatch Semi-SL training | Classifier training · More complex roll-out datasets | performance relative to ST and Self-SL | Hurt |
| Reduce BALD query size (ablation) | Self-SL models · CIFAR-100, low-label regime | accuracy relative to random querying | Helped |
| Reduce query size (ablation) | BALD with Self-SL · ISIC-2019, low-label regime | classification performance | No clear effect |
| Vary query size (ablation) | BADGE · CIFAR-100 and ISIC-2019, low-label regime | classification performance relative to random querying | No clear effect |
Closest papers by meaning
- Towards Robust and Reproducible Active Learning Using Neural Networks (CVPR 2022 · similarity 0.88)
- Towards Cost-Effective Learning: A Synergy of Semi-Supervised and Active Learning (CVPR 2025 · similarity 0.88)
- Composite Active Learning: Towards Multi-Domain Active Learning with Theoretical Guarantees (AAAI 2024 · similarity 0.88)
- A Cross-Domain Benchmark for Active Learning (NeurIPS 2024 · similarity 0.88)
- Instance-wise Supervision-level Optimization in Active Learning (CVPR 2025 · similarity 0.87)
- CoLAL: Co-learning Active Learning for Text Classification (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.