Research map / Adversarial robustness and security / Certified and empirical robustness

Wide Two-Layer Networks can Learn from Adversarial Perturbations

NeurIPS 2024 · Adversarial robustness and security › Certified and empirical robustness · approach: Attack evaluation and defense critique · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Theoretically explains why perturbation learning works by showing, for wide two-layer ReLU/Leaky-ReLU networks in the lazy training regime, that single-step adversarial perturbations align with weighted sums of training samples and that classifiers trained on perturbations or mislabeled adversarial examples agree with clean-trained classifiers.

Problem

Theoretical foundations of the feature hypothesis and perturbation learning (why classifiers trained only on mislabeled adversarial examples generalize to clean data) remain limited, and prior proofs rely on stringent assumptions.

Approach

Uses the lazy training regime for sufficiently wide two-layer networks to show that adversarial perturbations are approximately parallel to a kernel-weighted sum of training samples, and derives prediction-agreement conditions (functional margin and sign agreement) for any data distribution, with milder assumptions than prior orthogonality-based analysis.

Key results

• Prediction agreement between classifiers trained on clean data and on perturbations is observed experimentally on synthetic and real datasets when the theoretical conditions hold (e.g., d=100, N=1,000, m=100 verifies theorems on mean-shifted Gaussian data).
• Theorem shows the adversarial perturbation direction equals a weighted sum of training samples plus a remainder of norm O~(1), so alignment grows with input dimension d and training time T_f.
• Sufficient width m grows at rate d^2 times training-time terms; the paper reports m approximately 100 suffices for high-dimensional Gaussian data.

Measured effects reported in the paper

Surprising findings

• Perturbation learning succeeds even though classifiers are trained only on perturbations labeled opposite to human perception, and the theory shows this is expected under lazy training.
• Increasing perturbation size or sample size alone is not sufficient for prediction agreement; input dimension d is the consistently helpful factor.

Limitations

• Requires sufficiently wide (but finite) networks, with width growing as d^2 times training-time terms.
• Analysis restricted to two-layer networks, whereas practical perturbation learning uses deeper networks.
• Focuses on low-level features (the input itself) and does not address deep hierarchical features.

Experimental setup

Datasets: MNIST Fashion-MNIST synthetic zero-mean Gaussian synthetic mean-shifted Gaussian

Benchmarks: MNIST Fashion-MNIST synthetic Gaussian datasets

Metrics: accuracy agreement ratio cosine similarity

Baselines: Ilyas et al. feature learning analysis of perturbation learning

Builds on: feature hypothesis perturbation learning lazy training fast gradient sign / gradient-based single-step attack

Closest papers by meaning

• Theoretical Understanding of Learning from Adversarial Perturbations (ICLR 2024 · similarity 0.96)
• Defense Against Universal Adversarial Perturbations (CVPR 2018 · similarity 0.91)
• Lower bounds on the robustness to adversarial perturbations (NeurIPS 2017 · similarity 0.90)
• On Detecting Adversarial Perturbations (ICLR 2017 · similarity 0.90)
• Learn2Perturb: An End-to-End Feature Perturbation Learning to Improve Adversarial Robustness (CVPR 2020 · similarity 0.90)
• Adversarial Training Can Provably Improve Robustness: Theoretical Analysis of Feature Learning Process Under Structured Data (ICLR 2025 · similarity 0.89)