Research map / Label-efficient and robust learning / Recognition and long-tail
AutoEval: Are Labels Always Necessary for Classifier Accuracy Evaluation?
Weijian Deng, Liang Zheng
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace confidence-threshold prediction (tau=0.7) with neural network regression on dataset statistics | predicted score method · Digits (SVHN+USPS) | RMSE (%): 18.11 -> 1.46 | Improved |
| replace linear regression with neural network regression using mean and covariance features | linear regression · Digits (SVHN+USPS) | RMSE (%): 9.87 -> 1.46 | Improved |
| replace linear regression with neural network regression | linear regression · Natural images (Pascal+Caltech+ImageNet) | RMSE (%): 8.62 -> 3.04 | Improved |
| replace confidence-threshold prediction (tau=0.7) with neural network regression | predicted score method · Natural images (Pascal+Caltech+ImageNet) | RMSE (%): 1.49 -> 3.04 | Hurt |
| increase confidence threshold tau from 0.7 to 0.9 (ablation) | predicted score method · Natural images (Pascal+Caltech+ImageNet) | RMSE (%): 1.49 -> 6.96 | Hurt |
| increase confidence threshold tau from 0.7 to 0.9 (ablation) | predicted score method · Digits (SVHN+USPS) | RMSE (%): 18.11 -> 25.59 | Hurt |
| use neural network regression instead of linear regression on Caltech | linear regression · Caltech (12 classes, train COCO) | estimated accuracy (%): 79.77 -> 89.39 | Improved |
| use neural network regression instead of linear regression on USPS | linear regression · USPS (digits, train MNIST) | estimated accuracy (%): 50.14 -> 64.11 | Improved |
| increase number of sample sets in the training meta set (ablation) | neural network regression · Digits and natural images | absolute error (%) | Improved |
| increase number of sample sets in the training meta set (ablation) | linear regression · Digits and natural images | absolute error (%) | No clear effect |
| increase images per sample set (ablation) | neural network regression · Digits and natural images | absolute error (%) | Improved |
| increase images per sample set (ablation) | linear regression · Digits and natural images | absolute error (%) | No clear effect |
Closest papers by meaning
- AutoEval Done Right: Using Synthetic Data for Model Evaluation (ICML 2025 · similarity 0.87)
- Energy-based Automated Model Evaluation (ICLR 2024 · similarity 0.85)
- Pearls from Pebbles: Improved Confidence Functions for Auto-labeling (NeurIPS 2024 · similarity 0.84)
- Enhanced Sample Selection with Confidence Tracking: Identifying Correctly Labeled Yet Hard-to-Learn Samples in Noisy Data (AAAI 2025 · similarity 0.84)
- Doomed to Re-Annotate, Forever: The ImageNet Story (NeurIPS 2026 · similarity 0.84)
- Evaluating multiple models using labeled and unlabeled data (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
