Research map / Adversarial robustness and security / Certified and empirical robustness
Compositional Curvature Bounds for Deep Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Derives analytical, compositional upper bounds on the curvature (Lipschitz constant of the Jacobian) of sequential residual networks, yielding closed-form curvature-based robustness and attack certificates and a differentiable curvature regularizer.
Problem
Adversarial robustness of deep classifiers is certified only through first-order (Lipschitz) information; computing provable second-order (curvature) bounds for general sequential networks is missing and prior curvature methods were limited to scalar-valued, non-residual architectures.
Approach
A recursive compositional rule bounds the Lipschitz constant of the Jacobian layer by layer (combining LipLT-based Lipschitz bounds of subnetworks with per-block Jacobian bounds), enabling closed-form curvature-based certified and attack radii and a differentiable curvature regularizer; also introduces anchored Lipschitz constants.
Key results
- On CIFAR-10, the 6F model with CCRC regularization reaches 52.09% certified accuracy at eps=36/255 versus 42.73% for CRM, with 62.1% clean accuracy.
- On CIFAR-10, SLL + CCRC improves certified accuracy at eps=108/255 to 31.6% from SLL's 26.5%, at a small clean-accuracy cost (53.2% vs 57.2%).
- For a 6F CIFAR-10 model, attack certificates cover 808 samples, 645 of which require eps at most 36/255, tightening the verified accuracy range to [47.16, 49.74]%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CCRC instead of CRM | 6C2F · CIFAR-10, ε=36/255 | certified accuracy: 36.25 -> 49.53 | Helped |
| Use CCRC instead of CRM | 6C2F · CIFAR-10, ε=72/255 | certified accuracy: 18.36 -> 33.36 | Helped |
| Use CCRC instead of CRM | 6C2F · CIFAR-10, ε=108/255 | certified accuracy: 7.37 -> 16.95 | Helped |
| Use CCRC instead of CRM | 6F · CIFAR-10, ε=36/255 | certified accuracy: 42.73 -> 52.09 | Helped |
| Use CCRC instead of CRM | 6F · CIFAR-10, ε=72/255 | certified accuracy: 24.75 -> 40.8 | Helped |
| Use CCRC instead of CRM | 6F · CIFAR-10, ε=108/255 | certified accuracy: 12.6 -> 29.17 | Helped |
| Add CCRC curvature regularization | 6C2F · CIFAR-10 | accuracy: 58.57 -> 61.15 | Helped |
| Add CCRC curvature regularization | 6F · CIFAR-10 | accuracy: 60.63 -> 62.1 | Helped |
| Add CCRC curvature regularization to SLL | Lip-3C1F · CIFAR-10 | accuracy: 57.2 -> 53.2 | Hurt |
| Add CCRC curvature regularization to SLL | Lip-3C1F · CIFAR-10, ε=36/255 | certified accuracy: 45.0 -> 46.6 | Helped |
| Add CCRC curvature regularization to SLL | Lip-3C1F · CIFAR-10, ε=72/255 | certified accuracy: 35.0 -> 39.3 | Helped |
| Add CCRC curvature regularization to SLL | Lip-3C1F · CIFAR-10, ε=108/255 | certified accuracy: 26.5 -> 31.6 | Helped |
Surprising findings
- Curvature regularization reduced clean accuracy for the SLL Lip-3C1F model (57.2% to 53.2%), a trade-off the authors accept for higher certified accuracy at larger radii.
Limitations
- Bounds become conservative for deeper networks, which the authors identify as an open issue.
- Anchored bounds require numerical solution (bisection) of nonlinear equations per activation, and global bounds can be loose.
- Experiments are limited to MNIST and CIFAR-10 with fully connected and small convolutional networks.
Experimental setup
Datasets: MNIST CIFAR-10
Benchmarks: MNIST CIFAR-10
Metrics: certified accuracy certified radius attack certificate radius Lipschitz/curvature bound tightness
Baselines: CRM SLL Standard training Singla et al. curvature-based robustness certificate (CRC)
Builds on: LipLT LipSDP Singla et al. curvature bound
Closest papers by meaning
- Second-Order Provable Defenses against Adversarial Attacks (ICML 2020 · similarity 0.91)
- Robustness via Curvature Regularization, and Vice Versa (CVPR 2019 · similarity 0.90)
- Global curvature for second-order optimization of neural networks (ICML 2025 · similarity 0.87)
- Efficient Robust Conformal Prediction via Lipschitz-Bounded Networks (ICML 2025 · similarity 0.85)
- Sharper Guarantees for Learning Neural Network Classifiers with Gradient Methods (ICLR 2025 · similarity 0.85)
- ECLipsE: Efficient Compositional Lipschitz Constant Estimation for Deep Neural Networks (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.