Research map / Adversarial robustness and security / Certified and empirical robustness
RAMP: Boosting Adversarial Robustness Against Multiple $l_p$ Perturbations for Universal Robustness
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RAMP, a fine-tuning/AT framework combining a KL logit-pairing loss between l_q and l_r adversarial predictions with gradient projection from natural-training updates, to improve union accuracy against l1/l2/linf perturbations and universal robustness.
Problem
Adversarial training against a single l_p norm does not yield robustness to multiple l_p norms; the union accuracy (robust against l1, l2, linf jointly) is low, and there are tradeoffs among norms and between accuracy and robustness.
Approach
A KL logit-pairing loss that aligns l_r adversarial predictions with l_q predictions on correctly classified l_q examples, combined with a MAX-style worst-case loss, plus layer-wise gradient projection that injects useful natural-training update components into adversarial updates to moderate the accuracy/robustness tradeoff.
Key results
- Robust fine-tuning of a linf-AT WRN model reaches 53.3% union accuracy on CIFAR-10 and 29.1% on ImageNet (XCiT-S), versus E-AT's 51.6% and 23.4%.
- Training RN-18 from scratch on CIFAR-10 gives 44.6% union accuracy with 81.2% clean accuracy (lambda=5), above MAX's 44.0% union.
- On WRN-28-10 universal robustness, RAMP reaches 75.5% average common-corruption accuracy and 26.1% union accuracy against unseen adversaries, above E-AT (21.9%) and MAX (24.4%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use RAMP instead of E-AT | PreAct-ResNet-18 trained from scratch · CIFAR-10, (12, 0.5, 2/255) | union accuracy: 55.4 -> 59.1 | Helped |
| Use RAMP instead of MAX | PreAct-ResNet-18 trained from scratch · CIFAR-10, (12, 1.5, 8/255) | union accuracy: 35.6 -> 37.1 | Helped |
| Use RAMP instead of E-AT | PreAct-ResNet-18 with robust fine-tuning · CIFAR-10, (12, 1.5, 8/255) | union accuracy: 31.5 -> 36.9 | Helped |
| Use RAMP instead of E-AT | WRN-70-16 robust fine-tuning · CIFAR-10 | union accuracy: 51.6 -> 53.3 | Helped |
| Use RAMP instead of E-AT | XCiT-S robust fine-tuning · ImageNet | union accuracy: 23.4 -> 29.1 | Helped |
| Use RAMP instead of E-AT | PreAct-ResNet-18 trained from random initialization · CIFAR-10, AutoAttack | union accuracy: 42.4±0.6 -> 44.6±0.6 | Helped |
| Use RAMP instead of E-AT | WRN-28-10 · CIFAR-10 common corruptions | average accuracy: 49.4 -> 50.9 | Helped |
| Use RAMP instead of E-AT | WRN-28-10 · CIFAR-10 unseen adversaries | union accuracy: 21.9 -> 26.1 | Helped |
| Increase logit-pairing weight λ (ablation) | RAMP trained from scratch · CIFAR-10 | clean accuracy | Hurt |
| Set λ=2.0 for training from scratch (ablation) | RAMP · CIFAR-10 | clean and union accuracy | Mixed |
| Set λ=0.5 for robust fine-tuning (ablation) | RAMP · CIFAR-10 | clean and union accuracy | Mixed |
| Combine natural-training and adversarial-training updates with β=0.5 using gradient projection (ablation) | RAMP · CIFAR-10 ResNet-18 | clean and robust accuracy | Mixed |
Surprising findings
- Fine-tuning an linf-AT model on l1 examples or applying E-AT drops linf robustness substantially after just one epoch.
- RAMP has slightly lower clean accuracy than some baselines on several large models (e.g., 84.3 vs 85.3 for RN-50 CIFAR-10), and the paper notes occasional clean-accuracy drops during fine-tuning.
Limitations
- RAMP is more expensive than E-AT per epoch, though cheaper than MAX.
- Occasional drops in clean accuracy and single-norm robustness during fine-tuning.
- Key tradeoff pair and hyperparameters (lambda, beta) are selected heuristically per setting.
Experimental setup
Datasets: CIFAR-10 ImageNet
Benchmarks: CIFAR-10 ImageNet AutoAttack CIFAR-10-C common corruptions unseen adversaries (l0, fog, snow, gabor, elastic, jpeginf)
Metrics: clean accuracy l_inf accuracy l_2 accuracy l_1 accuracy union accuracy average corruption accuracy
Baselines: SAT AVG MAX MSD E-AT Winninghand
Builds on: adversarial training (PGD-AT) MAX (worst-case multi-norm AT) E-AT (extreme norm adversarial training) APGD AutoAttack
Closest papers by meaning
- Adversarial Robustness Limits via Scaling-Law and Human-Alignment Studies (ICML 2024 · similarity 0.87)
- Efficient Robustness Evaluation via Constraint Relaxation (AAAI 2025 · similarity 0.87)
- ROME is Forged in Adversity: Robust Distilled Datasets via Information Bottleneck (ICML 2025 · similarity 0.87)
- Robust SAM: On the Adversarial Robustness of Vision Foundation Models (AAAI 2025 · similarity 0.87)
- Expressive Losses for Verified Robustness via Convex Combinations (ICLR 2024 · similarity 0.87)
- Quadratic Upper Bound for Boosting Robustness (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.