Research map / Adversarial robustness and security / Certified and empirical robustness
Unveiling AI's Blind Spots: An Oracle for In-Domain, Out-of-Domain, and Adversarial Errors
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Trains a ViT or ResNet50 "mentor" network to predict whether a frozen mentee classifier will be correct on in-domain, out-of-domain, and adversarial images, finding adversarial-trained mentors work best and proposing the ViT-based SuperMentor.
Problem
Predicting when an image classifier (mentee) will make mistakes on in-domain, out-of-domain, or adversarial inputs, without relying on hand-crafted confidence or OOD metrics.
Approach
A separate mentor network, trained with a distillation loss to mimic the mentee's logits plus a binary correctness loss, learns the mentee's error patterns; training on small-perturbation adversarial (PIFGSM) errors yields the best generalization, and a ViT mentor (SuperMentor) is the strongest variant.
Key results
- SuperMentor achieves 70.4% average error-prediction accuracy on ImageNet-1K with a ResNet50 mentee, beating the best baseline TrustScore-level and SSL (66.7%) results.
- Transformer (ViT) mentors outperform ResNet50 mentors; e.g., C10 adversarial-attack error prediction for a ResNet50 mentee rises from 61.9% (R-R) to 73.1% (R-V).
- Mentor accuracy on C10-AA-PIFGSM drops from 74.3% at epsilon 1/255 to 49.7% at epsilon 8/255.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train the mentor on adversarial attack errors instead of ID or OOD errors | Mentor error prediction · CIFAR-10, CIFAR-100, ImageNet-1K | accuracy | Helped |
| Use a ViT rather than a ResNet50 mentor backbone | Mentor error prediction · CIFAR-10, CIFAR-100, ImageNet-1K | accuracy: 61.9% -> 73.1% | Helped |
| Train with smaller adversarial perturbations | PIFGSM-trained mentor · C10-AA-PIFGSM | accuracy: 49.7% -> 74.3% | Helped |
| Use smaller Speckle Noise distortions | SpN-trained mentor · C10-OOD-SpN and C100-OOD-SpN | accuracy: 47.9% -> 66.2% | Helped |
| Add ID and SpN data to PIFGSM training data (ablation) | C10 mentor · C10, average across error sources | accuracy: 74.3% -> 77.5% | Helped |
| Train jointly on all 9 ID, OOD, and AA error sources rather than PIFGSM alone (ablation) | C10 mentor · C10, average across error sources | accuracy: 74.3% -> 78.4% | Helped |
| Remove distillation loss L_d (ablation) | Full mentor · C10, average accuracy | accuracy: 64.9% -> 59.3% | Hurt |
| Replace distillation loss L_d with alignment loss L_a (ablation) | Full mentor · CIFAR-10, CIFAR-100, ImageNet-1K | accuracy | Hurt |
| Train only on adversarial examples with small perturbations | SuperMentor · ImageNet-1K AA-PGD | accuracy: 64.7% -> 73.1% | Helped |
Surprising findings
- Mentors trained only on adversarial-attack errors generalize to in-domain and OOD error prediction, outperforming mentors trained on ID or OOD errors.
- ResNet50- and ViT-based mentees produce similar error patterns, so mentors generalize across mentee architectures.
Limitations
- Evaluation restricted to image classification with ResNet50 and ViT mentees on CIFAR and ImageNet-scale datasets.
- Assumes white-box access to mentee parameters for generating adversarial error data.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 ImageNet-1K ImageNet-C corruptions NCTCRCHE100K ImageNet9 ImageNet-R ImageNet-Sketch
Benchmarks: CIFAR-10 CIFAR-100 ImageNet-1K NCTCRCHE100K ImageNet9 ImageNet-R ImageNet-Sketch
Metrics: error prediction accuracy (%) average accuracy across error sources
Baselines: Self Error Rate Maximum Class Probability Class Probability Entropy Distance To Centroid ConfidNet TrustScore Steep Slope Loss
Builds on: knowledge distillation PGD CW attack PIFGSM ConfidNet
Closest papers by meaning
- Failures Are Fated, But Can Be Faded: Characterizing and Mitigating Unwanted Behaviors in Large-Scale Vision and Language Models (ICML 2024 · similarity 0.84)
- Doppelgangers and Adversarial Vulnerability (CVPR 2025 · similarity 0.84)
- Human and AI Perceptual Differences in Image Classification Errors (AAAI 2025 · similarity 0.84)
- Quantifying Uncertainty in Error Consistency: Towards Reliable Behavioral Comparison of Classifiers (NeurIPS 2025 · similarity 0.83)
- Evaluating Model Bias Requires Characterizing its Mistakes (ICML 2024 · similarity 0.83)
- Model Diagnosis and Correction via Linguistic and Implicit Attribute Editing (CVPR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.