Research map / Adversarial robustness and security / Certified and empirical robustness
Interpreting Robustness Proofs of Deep Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SuPFEx, a greedy algorithm that extracts a small sufficient set of penultimate-layer proof features from DNN robustness proofs and uses gradient maps to show that standard, PGD, CROWN-IBP and COLT-trained networks rely on different (spurious vs. semantic) input features.
Problem
Robustness proofs produced by DNN verifiers capture invariants over infinite input regions, but it is unclear whether these proofs rely on human-understandable input features.
Approach
Treat the proof as a function of penultimate-layer interval proof features; define sufficiency and priority via pruning final-layer weight columns, and use a greedy halving search (SuPFEx) with an upper-bound priority to find small sufficient feature sets, then visualize their averaged input gradients.
Key results
- On a PGD-trained MNIST network, the Theorem-based baseline gives mean proof feature size 218.02 while SuPFEx gives 5.57 (median 3).
- On MNIST at eps=0.02, 100% of PGD-trained and COLT-trained networks have the same top feature under DeepZ and IBP; CROWN-IBP gives 99.79%.
- Qualitatively, standard network proofs rely on spurious background features, CROWN-IBP proofs filter out spurious features but miss some meaningful ones, and COLT proofs highlight relevant object features.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SuPFEx instead of the theorem-bound baseline to extract sufficient proof features | DNN robustness proofs · MNIST and CIFAR-10; settings in SuPFEx Efficacy Analysis | mean proof feature count: 23.19 -> 2.23; 218.02 -> 5.57; 44.43 -> 7.37; 42.38 -> 5.84; 279.31 -> 5.29; 51.01 -> 5.97; 47.92 -> 5.86; 52.93 -> 10.3 | Helped |
| Train with standard training rather than robust training | DNN robustness proofs · MNIST and CIFAR-10 | semantic reliance on input features | Mixed |
| Use PGD adversarial training rather than standard training | DNN robustness proofs · MNIST and CIFAR-10 | filtering of spurious features | Helped |
| Use CROWN-IBP certified training rather than standard training | DNN robustness proofs · MNIST and CIFAR-10 | filtering of spurious features | Mixed |
| Combine empirical and certified robustness training with COLT | DNN robustness proofs · MNIST and CIFAR-10 | semantic alignment of proof features | Helped |
| Increase training perturbation ε_train | PGD- and COLT-trained networks · MNIST | number of input features highlighted by top proof feature | Helped |
| Use IBP instead of DeepZ | Robustness proof generation · MNIST and CIFAR-10; PGD, COLT, and CROWN-IBP networks | percentage of properties proved: 82.0 % -> 26.0 %; 89.4 % -> 49.8 %; 96.4 % -> 93.4 %; 47.0 % -> 10.2 %; 53.0 % -> 17.2 %; 53.0 % -> 21.8 % | Hurt |
| Use larger ε for the input region φ | SuPFEx · MNIST and CIFAR-10 | proof feature extraction performance | No clear effect |
Surprising findings
- Certifiably trained (CROWN-IBP) proofs filter out spurious features but also drop some semantically meaningful ones.
Limitations
- Scalability is limited by existing verifiers, so ImageNet-scale networks cannot be handled.
- Results are based on small convolutional networks on MNIST and CIFAR-10 only.
- Algorithm yields approximate (not guaranteed minimum) sufficient feature sets.
Experimental setup
Datasets: MNIST CIFAR-10
Benchmarks: MNIST CIFAR-10
Metrics: proof feature set size (mean, median) percentage of proofs with <=5 and <=10 features agreement of top feature across verifiers
Baselines: Trivial zero-feature pruning Theorem upper bound on feature set size IBP verifier
Builds on: DeepZ CROWN-IBP COLT PGD adversarial training IBP
Closest papers by meaning
- Towards Trustworthy Deep Learning (AAAI 2024 · similarity 0.91)
- PROVEN: Verifying Robustness of Neural Networks with a Probabilistic Approach (ICML 2019 · similarity 0.90)
- VNN: Verification-Friendly Neural Networks with Hard Robustness Guarantees (ICML 2024 · similarity 0.90)
- Evaluating Robustness of Neural Networks with Mixed Integer Programming (ICLR 2019 · similarity 0.90)
- Learning Better Certified Models from Empirically-Robust Teachers (NeurIPS 2026 · similarity 0.89)
- Theoretical Analysis of Robust Overfitting for Wide DNNs: An NTK Approach (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.