Research map / Adversarial robustness and security / Certified and empirical robustness
Using Pre-Training Can Improve Model Robustness and Uncertainty
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add untargeted adversarial pre-training before adversarial fine-tuning | Wide ResNet · CIFAR-10 | adversarial accuracy: 45.8 -> 57.4 | Helped |
| Add untargeted adversarial pre-training before adversarial fine-tuning | Wide ResNet · CIFAR-100 | adversarial accuracy: 24.3 -> 33.5 | Helped |
| Pre-train before label-noise correction | Forward Correction · CIFAR-10 | error-curve AUC: 25.5 -> 15.7 | Helped |
| Pre-train before label-noise correction | GLC (5% Trusted) · CIFAR-10 | error-curve AUC: 14.0 -> 7.2 | Helped |
| Pre-train before label-noise correction | GLC (10% Trusted) · CIFAR-100 | error-curve AUC: 38.9 -> 28.4 | Helped |
| Pre-train on Downsampled ImageNet, then fine-tune on imbalanced data | Wide ResNet · CIFAR-10, imbalance ratio 0.2 | total test error rate: 19.7 -> 8.0 | Helped |
| Pre-train on Downsampled ImageNet, then fine-tune on imbalanced data | Wide ResNet · CIFAR-100, imbalance ratio 0.2 | minority test error rate: 61.0 -> 41.8 | Helped |
| Pre-train then fine-tune | Wide ResNet · CIFAR-100 OOD detection | AUROC: 69.4 -> 83.1 | Helped |
| Pre-train then fine-tune | Wide ResNet · CIFAR-10 | RMS calibration error: 6.4 -> 2.9 | Helped |
| Pre-train then fine-tune | Wide ResNet · CIFAR-100 | RMS calibration error: 13.3 -> 3.6 | Helped |
| Use pre-training with temperature tuning | Wide ResNet · Tiny ImageNet | RMS calibration error: 4.15 to 3.55 | Helped |
| Exclude CIFAR-10-related classes from Downsampled ImageNet pre-training (ablation) | Pre-Training · CIFAR-10 adversarial robustness | adversarial accuracy: 1.04% decrease | Hurt |
Closest papers by meaning
- Adversarial Robustness: From Self-Supervised Pre-Training to Fine-Tuning (CVPR 2020 · similarity 0.92)
- On Transfer of Adversarial Robustness from Pretraining to Downstream Tasks (NeurIPS 2023 · similarity 0.91)
- Robustness May Be at Odds with Accuracy (ICLR 2019 · similarity 0.90)
- Adversarial Training Can Provably Improve Robustness: Theoretical Analysis of Feature Learning Process Under Structured Data (ICLR 2025 · similarity 0.89)
- SURE: SUrvey REcipes for building reliable and robust deep networks (CVPR 2024 · similarity 0.88)
- Understanding and Mitigating the Label Noise in Pre-training on Downstream Tasks (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.