Research map / Adversarial robustness and security / Certified and empirical robustness
Explaining the role of Intrinsic Dimensionality in Adversarial Training
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Explains differing robustness/generalization effects of adversarial training across vision, encoder LLM and decoder LLM architectures via intrinsic dimensionality and the manifold conjecture, and proposes SMAAT, which adversarially perturbs the lowest-ID layer to cut AT cost while improving robustness.
Problem
Adversarial training affects robustness and generalization differently across vision models, encoder-based LLMs and decoder-based LLMs, and standard AT is computationally expensive due to multi-step PGD chains.
Approach
Uses layerwise intrinsic dimensionality (TwoNN) and the manifold conjecture to explain why off-manifold AEs favor robustness and on-manifold AEs favor generalization; SMAAT perturbs the intermediate layer with the lowest ID, which for encoder LLMs is the last layer, shortening backprop chains.
Key results
- SMAAT improves BERT robustness over the best baseline by 8.6%, 15.7%, and 28.8% on AGNEWS, IMDB, and YELP (RoBERTa: 6.0%, 5.8%, 19.0%) while keeping clean accuracy near standard training.
- SMAAT reduces training time to about 5.2 min/epoch versus 25.7 for ASCC and 15.6 for FreeLB++ on IMDB, roughly 3x faster than FreeLB++, and the abstract reports 25-33% of AT GPU time.
- On the Contriever RAG retriever (Natural Questions, N=50 poisoning passages), SMAAT achieves 85.7 robust recall@10 versus 26.1 for standard training.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Perturb the last layer instead of the input layer | BERT and RoBERTa classifiers · AGNEWS, IMDB, and YELP | average robust accuracy (AR): BERT: 8.6%, 15.7%, and 28.8%; RoBERTa: 6.0%, 5.8%, and 19.0% over the best score | Helped |
| Perturb the last layer instead of the input layer | BERT and RoBERTa classifiers · AGNEWS, IMDB, and YELP | clean accuracy: 0.5 | Mixed |
| Perturb the last layer instead of the input layer | BERT, RoBERTa, and DistilBERT safety filters · AdvBench and HH-RLHF | AUA: BERT: 100 vs 0 (AdvBench), 51.6 vs 48.5 (HH-RLHF); RoBERTa: 99.2 vs 0, 96.8 vs 34.0; DistilBERT: 97.5 vs 0, 39.5 vs 34.1 | Helped |
| Adversarially train by perturbing the selected low-ID layer | Contriever · Natural Questions poisoning, N=10 and N=50 | robust recall: N=10 R@10: 99.5 vs 75.2; N=50 R@10: 85.7 vs 51.7 | Helped |
| Adversarially train by perturbing the selected low-ID layer | BERT and RoBERTa · GLUE | average accuracy: BERT: 86.3 vs 86.3; RoBERTa: 89.7 vs 89.6 | No clear effect |
| Adversarially train by perturbing the selected low-ID layer | BERT and RoBERTa · AdvGLUE | average accuracy: BERT: 45.1 vs 39.5; RoBERTa: 39.6 vs 27.5 | Helped |
| Perturb the last layer to shorten PGD backpropagation chains | BERT on IMDB · IMDB | training time (min/epoch): 15.6 ± 0.5 -> 5.2 ± 0.2 | Helped |
| Apply AT at lower rather than upper layers | LLaMA-2-7B-chat · LAT dataset, MT-Bench and GCG | robustness | Mixed |
| Apply AT at higher rather than lower layers | BERT · YELP under TextFooler | robustness | Helped |
| Apply AT at lower rather than higher layers | VGG-13 · CIFAR-10 | robustness | Mixed |
Surprising findings
- Vision models and decoder LLMs show the highest off-manifold ratio in early layers rather than the first layer, an unexpected pattern the authors attribute to CLS/last-token embeddings needing time to reflect perturbations.
- FreeLB++ perturbing the first layer gives the best generalization on several tasks, yet SMAAT's last-layer perturbation keeps generalization nearly unchanged while improving robustness.
Limitations
- For vision and decoder LLMs the lowest-ID layer is the first layer, so SMAAT reduces to standard AT and gives no speed-up there.
- Experiments use a subset of YELP (63K samples) and 1000-sample test subsets due to compute limits.
- Lacks a formal proof linking ID to the on/off-manifold ratio; the relation is supported by empirical correlation and a heuristic argument.
Experimental setup
Datasets: AGNEWS IMDB YELP AdvBench HH-RLHF Natural Questions CIFAR-10 HelpSteer LAT dataset UltraChat GLUE AdvGLUE
Benchmarks: AGNEWS IMDB YELP AdvBench HH-RLHF Natural Questions GLUE AdvGLUE MT-Bench CIFAR-10
Metrics: clean accuracy robust accuracy under attack average robust accuracy robust recall recall@10 recall@100 training time per epoch inference time per sample
Baselines: ASCC FreeLB++ SAFER TMD RSMI FreeLB++ (retriever)
Builds on: projected gradient descent adversarial training latent adversarial training FreeLB TwoNN intrinsic dimension estimator
Closest papers by meaning
- Disentangling Adversarial Robustness and Generalization (CVPR 2019 · similarity 0.90)
- Enhancing the Adversarial Robustness via Manifold Projection (AAAI 2025 · similarity 0.89)
- Adversarial Training Can Provably Improve Robustness: Theoretical Analysis of Feature Learning Process Under Structured Data (ICLR 2025 · similarity 0.88)
- Rethinking Invariance Regularization in Adversarial Training to Improve Robustness-Accuracy Trade-off (ICLR 2025 · similarity 0.88)
- On the Duality Between Sharpness-Aware Minimization and Adversarial Training (ICML 2024 · similarity 0.88)
- Revisiting Adversarial Training at Scale (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.