Research map / Learning theory and algorithms / Prediction and decision losses
PAC Confidence Predictions for Deep Neural Network Classifiers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply temperature scaling instead of naive softmax | ResNet101 confidence predictions · ImageNet validation | ECE: 4.79% -> 1.66% | Helped |
| Use histogram binning instead of naive softmax | ResNet101 confidence predictions · ImageNet validation | ECE: 4.79% -> 0.99% | Helped |
| Use PAC confidence bounds to construct confidence intervals | Histogram-binned ResNet101 predictions · ImageNet validation | ECE interval coverage: [0.0%, 3.76%] | Helped |
| Use PAC confidence bounds to set the cascade exit threshold | ResNet101 cascading classifier · ImageNet test | inference time: 32% | Helped |
| Use PAC confidence bounds to set the cascade exit threshold | ResNet101 cascading classifier · ImageNet test | CPU/GPU inference time: 54% | Helped |
| Use PAC confidence bounds rather than bin means for cascade thresholding | ResNet101 cascading classifier · ImageNet test | desired error bound satisfaction | Helped |
| Use histogram bin means rather than PAC confidence bounds for cascade thresholding | ResNet101 cascading classifier · ImageNet test | inference time | Helped |
| Use PAC confidence bounds to set the shield threshold | AI Habitat recoverability shield · AI Habitat | safe rollout rate | Helped |
| Use histogram bin means rather than PAC confidence bounds for shield thresholding | AI Habitat recoverability shield · AI Habitat | success rate | No clear effect |
| Use a naive fixed threshold instead of the PAC-selected shield threshold | AI Habitat recoverability shield · AI Habitat | safety constraint satisfaction | Hurt |
| Set the shield threshold to ξ instead of selecting it with PAC bounds | AI Habitat recoverability shield · AI Habitat | safety constraint satisfaction | Hurt |
| Increase the calibration sample count n (ablation) | ResNet101 cascading classifier · ImageNet test | inference speed | Helped |
Closest papers by meaning
- PAC Confidence Sets for Deep Neural Networks via Calibrated Prediction (ICLR 2020 · similarity 0.90)
- SURE: SUrvey REcipes for building reliable and robust deep networks (CVPR 2024 · similarity 0.87)
- Confidence-Aware Learning for Deep Neural Networks (ICML 2020 · similarity 0.86)
- Towards Trustworthy Deep Learning (AAAI 2024 · similarity 0.85)
- VNN: Verification-Friendly Neural Networks with Hard Robustness Guarantees (ICML 2024 · similarity 0.84)
- Enumerating Safe Regions in Deep Neural Networks with Provable Probabilistic Guarantees (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.