Research map / Label-efficient and robust learning / Recognition and long-tail
Are Labels Always Necessary for Classifier Accuracy Evaluation?
Weijian Deng, Liang Zheng
Check your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use neural network regression instead of linear regression | AutoEval · Digits (SVHN, USPS) | RMSE (%): 9.87 -> 1.46 | Improved |
| Use neural network regression instead of linear regression | AutoEval · Natural images (PASCAL, Caltech, ImageNet) | RMSE (%): 8.62 -> 3.04 | Improved |
| Increase the confidence threshold from τ=0.7 to τ=0.9 | Confidence score accuracy estimator · Natural images (PASCAL, Caltech, ImageNet) | RMSE (%): 1.49 -> 6.96 | Hurt |
| Increase the confidence threshold from τ=0.7 to τ=0.9 | Confidence score accuracy estimator · Digits (SVHN, USPS) | RMSE (%): 18.11 -> 25.59 | Hurt |
| Use confidence-threshold prediction instead of learned regression | AutoEval · Digits (SVHN, USPS) | RMSE (%): 1.46 vs 18.11 | Hurt |
| Train the neural regressor with more meta-dataset sample sets (ablation) | Neural network regression · Synthetic meta-dataset | Absolute prediction error | Improved |
| Increase the number of images in each meta-dataset sample set (ablation) | Neural network regression · Synthetic meta-dataset | Absolute prediction error | Improved |
| Train the linear regressor with fewer meta-dataset sample sets (ablation) | Linear regression · Synthetic meta-dataset | Absolute prediction error | No clear effect |
| Apply unseen image transformations to real-world test datasets (ablation) | Regression-based AutoEval · Edited PASCAL, Caltech, and ImageNet | Absolute prediction error | Mixed |
Closest papers by meaning
- AutoEval: Are Labels Always Necessary for Classifier Accuracy Evaluation? (TPAMI 2021 · similarity 0.99)
- AutoEval Done Right: Using Synthetic Data for Model Evaluation (ICML 2025 · similarity 0.86)
- Evaluating multiple models using labeled and unlabeled data (NeurIPS 2025 · similarity 0.84)
- Enhanced Sample Selection with Confidence Tracking: Identifying Correctly Labeled Yet Hard-to-Learn Samples in Noisy Data (AAAI 2025 · similarity 0.84)
- Energy-based Automated Model Evaluation (ICLR 2024 · similarity 0.84)
- Doomed to Re-Annotate, Forever: The ImageNet Story (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
