Research map / Adversarial robustness and security / Black-box and transfer attacks
Disrupting Deep Uncertainty Estimation Without Harming Accuracy
Ido Galil, Ran El-Yaniv
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add ACE black-box perturbations with ε=0.05 (ablation) | ResNet50 softmax confidence · ImageNet validation | AURC: 69.9 -> 456.6 | Hurt |
| Add ACE black-box perturbations with ε=0.05 (ablation) | ResNet50 softmax classifier · ImageNet validation | Accuracy: 76.01 -> 76.01 | No clear effect |
| Use white-box access rather than a black-box proxy (ablation) | EfficientNetB0 softmax uncertainty under ACE · ImageNet validation | AURC: 69.1 -> 553.3 | Hurt |
| Use a matching ResNet50 proxy rather than a foreign-architecture proxy (ablation) | ACE attacking a ResNet50 ensemble · ImageNet validation | AURC: 290.8 -> 381 | Hurt |
| Increase the ensemble size from 3 to 10 (ablation) | ResNet50 ensemble under white-box ACE · ImageNet validation | AURC: 533.4 -> 510.5 | No clear effect |
| Attack MC-Dropout using ACE with ε=0.05 (ablation) | MobileNetV2 MC-Dropout, 30 passes · ImageNet validation | AURC: 94.5 -> 275.2 | Hurt |
| Increase MC-Dropout passes from 10 to 30 under black-box ACE (ablation) | MobileNetV2 MC-Dropout · ImageNet validation | AURC: 275.3 -> 275.2 | No clear effect |
| Target softmax indirectly instead of targeting predictive entropy directly (ablation) | White-box ACE on MC-Dropout · ImageNet validation | AURC: 587.6 -> 622.9; 433.9 -> 502.9 | Mixed |
| Increase MC-Dropout passes from 10 to 30 for direct attacks (ablation) | White-box ACE targeting predictive entropy · ImageNet validation | AURC: 587.9 -> 587.6; 521.2 -> 433.9 | No clear effect |
| Attack softmax rather than variance directly (ablation) | MC-Dropout epistemic uncertainty · ImageNet validation | AURC: 624.3 vs 453.2; 565.9 vs 414.5 | Mixed |
| Add ACE directly to the SelectiveNet selector head at ε=0.005 (ablation) | SelectiveNet with ResNet18 backbone · CIFAR-10 test | Selective risk: 0.0012 -> 0.0747 | Hurt |
| Attack softmax indirectly rather than the selector directly (ablation) | SelectiveNet with VGG16 backbone, ε=0.005 · CIFAR-10 test | Selective risk: 0.0772 -> 0.0829 | Hurt |
Closest papers by meaning
- Bias-Reduced Uncertainty Estimation for Deep Neural Classifiers (ICLR 2019 · similarity 0.88)
- Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles (NeurIPS 2017 · similarity 0.87)
- SURE: SUrvey REcipes for building reliable and robust deep networks (CVPR 2024 · similarity 0.87)
- What Can we Learn From The Selective Prediction And Uncertainty Estimation Performance Of 523 Imagenet Classifiers? (ICLR 2023 · similarity 0.87)
- Robustness Meets Uncertainty: Evidential Adversarial Training for Robust Selective Classification (ECCV 2026 · similarity 0.87)
- Make Me a BNN: A Simple Strategy for Estimating Bayesian Uncertainty from Pre-trained Models (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
