Research map / Learning theory and algorithms / Prediction and decision losses
Understanding Certified Training with Interval Bound Propagation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces propagation tightness, a ratio between optimal and IBP box radii, to show theoretically that IBP training increases tightness in deep linear networks and empirically explains the success of IBP-based certified training.
Problem
Why do certified training methods based on interval bound propagation (IBP) outperform methods using tighter bounds, and what regularization do they induce?
Approach
Defines propagation tightness as the ratio of optimal box radius to layer-wise (IBP) box radius, derives necessary and sufficient conditions for IBP to be exact (propagation invariance), and shows that IBP training increases tightness while width/depth affect tightness at initialization differently; the increase is driven by the propagated input region size.
Key results
- Tightness at initialization decreases as Theta(1/sqrt(width)) and CNN7 penultimate layer tau(500) ~ 0.056.
- IBP training pushes tightness toward propagation invariance (tau ~ 0.98) for CNN3 on CIFAR-10 at large epsilon.
- Widening CNN7 improves certified accuracy for IBP, SABR and IBP-R, while deeper networks do not help; SABR with width 2x reaches 34.95% certified on CIFAR-10 eps=8/255.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase CNN7 width from 1× to 2× (ablation) | IBP training · CIFAR-10 ε=2/255 | certified accuracy | Helped |
| Increase CNN7 width from 1× to 2× (ablation) | IBP training · CIFAR-10 ε=8/255 | certified accuracy | Hurt |
| Increase CNN7 width from 1× to 4× (ablation) | IBP training · MNIST ε=0.1 | certified accuracy | Helped |
| Increase CNN7 width from 1× to 4× (ablation) | IBP training · MNIST ε=0.3 | certified accuracy | Helped |
| Increase network width (ablation) | IBP training · CIFAR-10 | training-set IBP-certified accuracy | Helped |
| Increase network depth beyond 7 layers (ablation) | CNN trained with IBP · CIFAR-10 ε=2/255 | training-set IBP-certified accuracy | Hurt |
| Train with IBP instead of PGD | CNN3 · CIFAR-10 | propagation tightness: τ=0.98 | Helped |
| Train with PGD instead of IBP | CNN3 · CIFAR-10 | propagation tightness | No clear effect |
| Reduce SABR subselection ratio λ (ablation) | SABR · CIFAR-10 | certified accuracy: λ=0.4 | Mixed |
| Increase the propagated input-region size ξ (ablation) | IBP-based training · CIFAR-10 | propagation tightness | Helped |
| Use the more strongly regularizing SABR rather than IBP-R | Certified training · CIFAR-10 | certified accuracy | Mixed |
Surprising findings
- Tightness is dominated by propagated region size xi = lambda*epsilon rather than the robustness specification epsilon itself.
- Non-IBP training (PGD, COLT) barely increases tightness, yet IBP-R and COLT remain much tighter than PGD.
Limitations
- Theory is for deep linear networks; ReLU results rely on local tightness approximation and transfer.
- Local tightness is an approximation that becomes less accurate for larger perturbations.
Experimental setup
Datasets: MNIST CIFAR-10 TinyImageNet
Benchmarks: MNIST CIFAR-10 TinyImageNet
Metrics: standard accuracy certified accuracy propagation tightness
Baselines: IBP SABR IBP-R PGD adversarial training COLT CROWN-IBP MTL-IBP TAPS
Builds on: interval bound propagation SABR IBP-R CROWN-IBP MN-BaB
Closest papers by meaning
- Certified Robustness for Deep Equilibrium Models via Interval Bound Propagation (ICLR 2022 · similarity 0.92)
- On the Convergence of Certified Robust Training with Interval Bound Propagation (ICLR 2022 · similarity 0.92)
- Towards Evaluating and Training Verifiably Robust Neural Networks (CVPR 2021 · similarity 0.92)
- Learning Better Certified Models from Empirically-Robust Teachers (NeurIPS 2026 · similarity 0.87)
- DeepBern-Nets: Taming the Complexity of Certifying Neural Networks Using Bernstein Polynomial Activations and Precise Bound Propagation (AAAI 2024 · similarity 0.87)
- Tightening Robustness Verification of MaxPool-based Neural Networks via Minimizing the Over-Approximation Zone (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.