Research map / Adversarial robustness and security / Certified and empirical robustness
Identifying and Understanding Cross-Class Features in Adversarial Training
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes cross-class features (features shared by multiple classes) as an explanation for AT dynamics, showing models lose them after the best checkpoint, causing robust overfitting, and that soft labels/KD preserve them.
Problem
The training dynamics and mechanisms behind adversarial training, especially robust overfitting and why soft-label methods such as knowledge distillation help AT, remain poorly understood.
Approach
Measures class-wise feature attribution similarity (cosine correlation matrix and a CAS metric) on adversarial examples to quantify use of cross-class robust features; hypothesizes that AT first learns and then forgets cross-class features, and that smoothed labels preserve them. Supported by a synthetic 3-class data model with theorems.
Key results
- Best robust checkpoint has higher CAS than last checkpoint (e.g., eps=8/255 CIFAR-10: CAS 25.6 at best vs 9.0 at last, RA 47.8% vs 42.5%).
- Adversarial training with larger epsilon shows a larger best-to-last CAS gap, consistent with more robust overfitting for epsilon up to 8/255.
- AT with knowledge distillation shows no significant CAS drop between best and last checkpoint, consistent with mitigated robust overfitting.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the best rather than last checkpoint of standard adversarial training | AT, CIFAR-10 ε=8/255 · CIFAR-10 | robust accuracy: 42.5% -> 47.8% | Helped |
| Use the best rather than last checkpoint of standard adversarial training | AT, CIFAR-10 ε=8/255 · CIFAR-10 | CAS: 9.0 -> 25.6 | Helped |
| Increase the perturbation bound from 8/255 to 16/255 | AT, epoch 10 · CIFAR-10 | CAS: 16.7 -> 14.4 | Hurt |
| Increase the perturbation bound from 8/255 to 16/255 | AT, best checkpoint · CIFAR-10 | robust accuracy: 47.8% -> 31.3% | Hurt |
| Train with knowledge distillation instead of one-hot labels | Adversarial training · CIFAR-10 | robust accuracy: 41% -> 48% | Helped |
| Use soft-label training such as knowledge distillation | Adversarial training · AT checkpoints | robust accuracy | Helped |
| Use a best rather than last checkpoint | AT on CIFAR-100 and Tiny-ImageNet · CIFAR-100; Tiny-ImageNet | CAS | Helped |
| Use a best rather than last checkpoint | ℓ2-norm AT · CIFAR-10 | CAS | Helped |
| Use a best rather than last checkpoint | DeiT-Ti adversarial training · CIFAR-10 | CAS | Helped |
| Use a best rather than post-catastrophic-overfitting checkpoint | Fast adversarial training · CIFAR-10 | CAS: 2.1 | Helped |
| Use a best rather than post-catastrophic-overfitting checkpoint | Fast adversarial training · CIFAR-10 | robust accuracy | Helped |
| Increase ε within the usual range up to 8/255 | Standard adversarial training · CIFAR-10 | robust overfitting | Hurt |
Surprising findings
- Extremely large epsilon (>8/255) suppresses cross-class feature learning early on, which the authors argue explains why robust overfitting weakens at very large epsilon.
Limitations
- Cross-class feature measure is a heuristic correlation metric on averaged attribution vectors rather than a direct feature-level analysis
- Theory relies on a simplified linear synthetic 3-class model with symmetry assumptions
- Instance-wise pairwise averaging was too noisy to be informative
Experimental setup
Datasets: CIFAR-10 CIFAR-100 Tiny-ImageNet
Benchmarks: CIFAR-10 CIFAR-100 Tiny-ImageNet
Metrics: robust accuracy Class Attribution Similarity (CAS)
Baselines: vanilla AT AT with knowledge distillation fast adversarial training
Builds on: PGD adversarial training knowledge distillation robust features
Closest papers by meaning
- Adversarial Training Can Provably Improve Robustness: Theoretical Analysis of Feature Learning Process Under Structured Data (ICLR 2025 · similarity 0.89)
- Theoretical Understanding of Learning from Adversarial Perturbations (ICLR 2024 · similarity 0.88)
- Wide Two-Layer Networks can Learn from Adversarial Perturbations (NeurIPS 2024 · similarity 0.88)
- Soften to Defend: Towards Adversarial Robustness via Self-Guided Label Refinement (CVPR 2024 · similarity 0.87)
- Cross-Class Feature Augmentation for Class Incremental Learning (AAAI 2024 · similarity 0.87)
- On the Clean Generalization and Robust Overfitting in Adversarial Training from Two Theoretical Views: Representation Complexity and Training Dynamics (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.