Research map / Adversarial robustness and security / Certified and empirical robustness
High-dimensional (Group) Adversarial Training in Linear Regression
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Gives non-asymptotic prediction-error bounds for l_infinity adversarial training of sparse high-dimensional linear regression (minimax-optimal up to log factor) and extends the analysis to group adversarial training under (2,infty) perturbations.
Problem
Statistical properties of adversarial training under l_infinity perturbations in high-dimensional sparse linear regression are not understood non-asymptotically; it is unknown whether the estimator attains minimax prediction error, and whether group structure in perturbations improves rates.
Approach
Uses the dual of l_infinity adversarial linear regression (a LASSO-like l1-regularized objective with a squared-loss-dependent term) and a restricted-eigenvalue argument to show prediction error of order s log p / n; extends to group perturbations under a (2,infinity) norm with group restricted eigenvalue condition, giving rate (|G_J| + g log L)/n.
Key results
- Under RE(s,3) and a perturbation level delta of order sqrt(log p / n), the l_infinity adversarial training prediction error satisfies (1/2n)||X(beta_hat - beta*)||^2 <= 192 s log p / n * max{...}, i.e. order s log p / n, minimax optimal up to log factor.
- Group adversarial training under (2,infinity) perturbation yields prediction error of order (|G_J| + g log L)/n, which beats classic adversarial training when |G_J|/s << log p and g << s.
- In simulations with p=500, log-log slopes of prediction error versus n are approximately -1 for both methods, and group adversarial training has lower error.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ℓ∞ adversarial training with perturbation magnitude of order 1/√n | High-dimensional sparse linear regression · High-dimensional sparse linear regression | prediction error rate: s\log p/n | Helped |
| Use group-structured (2,∞) perturbations instead of classic ℓ∞ perturbations | Adversarial training · Group-sparse linear regression with |G_J|/s ≪ log p and g ≪ s | prediction error upper bound | Helped |
| Use group-structured (2,∞) perturbations | Adversarial training · Synthetic data with 500-dimensional parameter divided into 125 groups of size 4 | prediction error | Helped |
Limitations
- Analysis restricted to linear regression with fixed design and Gaussian noise; no estimation-error or variable-selection guarantees yet
- Requires the perturbation level delta to be chosen using knowledge of the noise scale and sparsity; group analysis assumes X_{G_l}^T X_{G_l}/n = I
- Rate bounds hold only with high probability and involve constants and a log p factor
Experimental setup
Datasets: synthetic Gaussian linear regression data
Benchmarks: synthetic linear regression simulations
Metrics: prediction error log-log slope of prediction error vs sample size coefficient estimation path
Baselines: classic l_infinity adversarial training LASSO group LASSO
Builds on: LASSO group LASSO adversarial training dual formulation (Ashlagi-style l_infinity linear regressio restricted eigenvalue condition
Closest papers by meaning
- GSE: Group-wise Sparse and Explainable Adversarial Attacks (ICLR 2025 · similarity 0.85)
- Consistent Adversarially Robust Linear Classification: Non-Parametric Setting (ICML 2024 · similarity 0.85)
- Adversarial Training Can Provably Improve Robustness: Theoretical Analysis of Feature Learning Process Under Structured Data (ICLR 2025 · similarity 0.85)
- Kernel Learning with Adversarial Features: Numerical Efficiency and Adaptive Regularization (NeurIPS 2025 · similarity 0.85)
- Transferable Structural Sparse Adversarial Attack Via Exact Group Sparsity Training (CVPR 2024 · similarity 0.85)
- Clarifying the Behavior and the Difficulty of Adversarial Training (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.