Research map / Adversarial robustness and security / Certified and empirical robustness
Adversarial generalization of unfolding (model-based) networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Derives adversarial generalization bounds (via adversarial Rademacher complexity over FGSM-perturbed inputs) for an overparameterized ADMM-based unfolding network (ADMM-DAD) for compressed sensing, with experiments on CIFAR10 and SVHN.
Problem
Theoretical understanding of adversarial generalization of deep unfolding networks (DUNs) for compressed sensing under l2-norm FGSM attacks is lacking.
Approach
Proves Lipschitz continuity of the FGSM-perturbed ADMM-DAD decoder with respect to the shared overcomplete sparsifier, and uses it with covering numbers and Dudley's inequality to bound the adversarial Rademacher complexity, yielding a generalization bound scaling like sqrt(NL log(1+eps)).
Key results
- On CIFAR10 with 10-layer ADMM-DAD (N=10n), adversarial EGE grows from 0.18e-4 at eps=0.01 to 1.16e-4 at eps=1, consistent with the sqrt(L log(1+eps)) theory.
- Increasing overcompleteness N lowers both clean and adversarial test MSE for fixed eps, supporting overparameterization as a robustness aid.
- ADMM-DAD (N=10n) outperforms the 10-layer ISTA-Net baseline on CIFAR10 and SVHN in clean MSE, adversarial MSE and adversarial EGE at all tested attack levels.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use an overcomplete sparsifier (ADMM-DAD, N=10n) instead of an orthogonal sparsifier (ISTA-net) | 10-layer compressed-sensing DUN · CIFAR10, FGSM ε=0.01, 0.1, 1 | clean test MSE: 0.021 -> 0.019; 0.027 -> 0.023; 0.229 -> 0.026 | Helped |
| Use an overcomplete sparsifier (ADMM-DAD, N=10n) instead of an orthogonal sparsifier (ISTA-net) | 10-layer compressed-sensing DUN · CIFAR10, FGSM ε=0.01, 0.1, 1 | adversarial test MSE: 0.023 -> 0.020; 0.047 -> 0.025; 0.229 -> 0.034 | Helped |
| Use an overcomplete sparsifier (ADMM-DAD, N=10n) instead of an orthogonal sparsifier (ISTA-net) | 10-layer compressed-sensing DUN · CIFAR10, FGSM ε=0.01, 0.1, 1 | adversarial EGE: 0.70\cdot10^{-2} -> 0.18\cdot10^{-4}; 0.55\cdot10^{-2} -> 0.41\cdot10^{-4}; 0.15\cdot10^{-2} -> 1.16\cdot10^{-4} | Helped |
| Use an overcomplete sparsifier (ADMM-DAD, N=10n) instead of an orthogonal sparsifier (ISTA-net) | 10-layer compressed-sensing DUN · SVHN, FGSM ε=0.01, 0.1, 1 | clean test MSE: 0.028 -> 0.013; 0.036 -> 0.014; 0.676 -> 0.025 | Helped |
| Use an overcomplete sparsifier (ADMM-DAD, N=10n) instead of an orthogonal sparsifier (ISTA-net) | 10-layer compressed-sensing DUN · SVHN, FGSM ε=0.01, 0.1, 1 | adversarial test MSE: 0.030 -> 0.013; 0.054 -> 0.015; 0.703 -> 0.039 | Helped |
| Use an overcomplete sparsifier (ADMM-DAD, N=10n) instead of an orthogonal sparsifier (ISTA-net) | 10-layer compressed-sensing DUN · SVHN, FGSM ε=0.01, 0.1, 1 | adversarial EGE: 0.89\cdot10^{-2} -> 0.22\cdot10^{-4}; 0.78\cdot10^{-2} -> 3.77\cdot10^{-4}; 15\cdot10^{-2} -> 5.87\cdot10^{-4} | Helped |
| Increase sparsifier overcompleteness N | ADMM-DAD · CIFAR10 and SVHN, fixed FGSM attack level | clean and adversarial test MSE | Helped |
| Increase attack level ε | ADMM-DAD · CIFAR10 and SVHN, FGSM and PGD | adversarial test MSE | Hurt |
| Increase the number of layers L and attack level ε | ADMM-DAD · CIFAR10 and SVHN, FGSM and PGD | adversarial EGE | Hurt |
Surprising findings
- The robustness gap of ADMM-DAD is especially small (~1e-4) at eps=0.01, and the PGD-attacked network scales similarly to the theory derived for FGSM.
Limitations
- Theory is derived only for FGSM (l2) attacks; PGD is only evaluated empirically.
- Bounds assume boundedness of inputs/outputs and nonvanishing attack gradient (Assumption c), and the soft-thresholding derivative is handled nonsmoothly.
- Tightness of the bounds with respect to overparameterization is not established.
Experimental setup
Datasets: CIFAR10 SVHN
Benchmarks: CIFAR10 SVHN
Metrics: clean test MSE adversarial test MSE adversarial empirical generalization error robustness gap
Baselines: ISTA-Net
Builds on: ADMM-DAD ADMM ISTA-Net Dudley's inequality
Closest papers by meaning
- Sparsity Winning Twice: Better Robust Generalization from More Efficient Training (ICLR 2022 · similarity 0.88)
- Towards Deep Learning Models Resistant to Adversarial Attacks (ICLR 2018 · similarity 0.87)
- Certified Robust Neural Networks: Generalization and Corruption Resistance (ICML 2023 · similarity 0.87)
- Adversarial Robust Generalization of Graph Neural Networks (ICML 2025 · similarity 0.87)
- Stability and Generalization of Adversarial Training for Shallow Neural Networks with Smooth Activation (NeurIPS 2024 · similarity 0.87)
- If You Want to Be Robust, Be Wary of Initialization (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.