Research map / Adversarial robustness and security / Certified and empirical robustness
ImageNet Pre-training Also Transfers Non-robustness
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Initialize from ImageNet and fully fine-tune instead of training from scratch | ResNet-18 classifiers · Pets, NICO, Flowers, Cars, Food, CIFAR10, Alphabet | DR: 33.63 -> 82.51 (Pets); 34.24 -> 69.95 (NICO); 8.69 -> 69.71 (Flowers); 28.07 -> 77.14 (Cars); 62.10 -> 71.74 (Food); 39. | Hurt |
| Fine-tune all layers instead of only the classifier (ablation) | ImageNet-pre-trained model · Pets, NICO, Flowers, Cars, Food, CIFAR10, Alphabet | DR: 96.09 -> 82.51 (Pets); 88.64 -> 69.95 (NICO); 90.64 -> 69.71 (Flowers); 99.76 -> 77.14 (Cars); 98.73 -> 71.74 (Food); 10 | Helped |
| Increase pre-training model capacity (ablation) | ImageNet-pre-trained fine-tuned model · All 7 datasets (average) | DR: 81.47 -> 69.12 | Helped |
| Use a source dataset with fewer semantics (CIFAR10 rather than ImageNet-10animals) (ablation) | Fine-tuned model · Pets and NICO | DR: 77.56 -> 57.85 (Pets); 53.08 -> 38.63 (NICO) | Helped |
| Adversarially train at pre-training and fine-tuning, then regularize feature-space steepness | ImageNet transfer learning · Pets, NICO, Flowers, CIFAR10, Alphabet | AAI: 15.7 -> 77.73 (Pets); 28.33 -> 85.50 (NICO); 27.86 -> 81.41 (Flowers); 1.34 -> 88.63 (CIFAR10); 2.90 -> 99.90 (Alphabet) | Helped |
| Use discrepancy-mitigating steepness regularization at both stages | ImageNet transfer learning · Cars and Food | AAI: 18.57 vs 59.02 (Cars); 22.30 vs 58.48 (Food) | Mixed |
| Use a larger-capacity ImageNet pre-training backbone (ablation) | Fine-tuned model · All 7 datasets (average) | AAI: 16.71 -> 29.64 | Helped |
| Use ImageNet-10animals rather than CIFAR10 for source pre-training (ablation) | Fine-tuned model · Pets and NICO | AOI: 62.85 -> 75.91 (Pets); 77.76 -> 88.90 (NICO) | Helped |
| Use ImageNet-pre-trained features rather than Alphabet-trained features | Feature extractor evaluated on Alphabet · Alphabet | Local Lipschitzness: 32.9 -> 367.4 | Hurt |
| Transfer features from the ImageNet pre-trained model into fine-tuning | Fine-tuned model · Downstream classification tasks | AAI under pre-trained-model UAP | Hurt |
Closest papers by meaning
- Do Adversarially Robust ImageNet Models Transfer Better? (NeurIPS 2020 · similarity 0.91)
- Adversarial Robustness: From Self-Supervised Pre-Training to Fine-Tuning (CVPR 2020 · similarity 0.89)
- On Transfer of Adversarial Robustness from Pretraining to Downstream Tasks (NeurIPS 2023 · similarity 0.89)
- On the Connection between Pre-training Data Diversity and Fine-tuning Robustness (NeurIPS 2023 · similarity 0.89)
- Initialization Matters for Adversarial Transfer Learning (CVPR 2024 · similarity 0.89)
- Adversarial Training Can Provably Improve Robustness: Theoretical Analysis of Feature Learning Process Under Structured Data (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.