Research map / Architectures and efficiency / RNNs and normalization
DeepBern-Nets: Taming the Complexity of Certifying Neural Networks Using Bernstein Polynomial Activations and Precise Bound Propagation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DeepBern-Nets, networks whose activations are learnable Bernstein polynomials, and Bern-IBP, an interval-bound-propagation certifier that yields tight output bounds for robustness certification and certified training.
Problem
Incomplete certifiers for ReLU networks give loose bounds that degrade with depth, so adversarially trained networks are hard to certify efficiently.
Approach
Replace ReLU with learnable Bernstein-polynomial activations whose range-enclosure and subdivision properties let interval-style bound propagation compute tight bounds cheaply, with bounds of each activation's input precomputed during training.
Key results
- On MNIST CNNa 4 at eps 0.01, Bern-IBP certifies 95.97% vs 0% for IBP (test accuracy 97.23%).
- On CIFAR-10 CNNb 4 at eps 1/255, Bern-IBP certifies 42.86% vs 0% for IBP, with each certification taking a few seconds.
- Certified training with Bern-IBP matches or exceeds SOK ReLU results in many settings (e.g., CNNa MNIST eps 0.3: 82% vs 88% SOK; CNNb CIFAR eps 8/255: 37% vs 35%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Bern-IBP instead of standard IBP | DeepBern-Nets · MNIST and CIFAR-10 adversarial robustness | certified accuracy (%): 0 -> 88.69; 0 -> 72.12; 0 -> 65.22; 0 -> 80.21; 0 -> 56.49; 0 -> 72.35; 0 -> 27.74; 0 -> 33.49; 0 -> 28.55; 0 -> 14.7 | Helped |
| Use Bernstein polynomial activations with Bern-IBP for certified training instead of the SOK ReLU setup | FCNNa · MNIST, ϵ=0.1 | certified accuracy (%): 68 -> 72 | Helped |
| Use Bernstein polynomial activations with Bern-IBP for certified training instead of the SOK ReLU setup | FCNNa · CIFAR-10, ϵ=8/255 | certified accuracy (%): 27 -> 28 | Helped |
| Use Bernstein polynomial activations with Bern-IBP for certified training instead of the SOK ReLU setup | FCNNa · CIFAR-10, ϵ=2/255 | certified accuracy (%): 33 -> 38 | Helped |
Surprising findings
- IBP bounds become exponentially looser as Bernstein activation order increases, while Bern-IBP bounds stay precise at higher orders.
Limitations
- Certified accuracy on some settings (e.g., CIFAR-10 at larger eps, some FCNN settings) is lower than or comparable to ReLU SOK results.
- Evaluation limited to small MNIST and CIFAR-10 networks plus one quadrotor reachability case.
- Bern-IBP rests on assumptions of bounded known input domain and polynomial order choice per neuron.
Experimental setup
Datasets: MNIST CIFAR-10
Benchmarks: MNIST CIFAR-10 SOK benchmark
Metrics: certified accuracy verification time lower bound on robustness volume error of reachable sets
Baselines: IBP (auto_LiRPA) alpha-CROWN CROWN CROWN-IBP (SOK benchmark)
Builds on: interval bound propagation CROWN-IBP PGD adversarial training Bernstein polynomial subdivision and enclosure properties
Closest papers by meaning
- Verifying Properties of Binary Neural Networks Using Sparse Polynomial Optimization (ICLR 2025 · similarity 0.87)
- Clip-and-Verify: Linear Constraint-Driven Domain Clipping for Accelerating Neural Network Verification (NeurIPS 2025 · similarity 0.87)
- Understanding Certified Training with Interval Bound Propagation (ICLR 2024 · similarity 0.87)
- Dynamic Back-Substitution in Bound-Propagation-Based Neural Network Verification (AAAI 2025 · similarity 0.86)
- Sharper Guarantees for Learning Neural Network Classifiers with Gradient Methods (ICLR 2025 · similarity 0.86)
- Certified but Private: Scalable Zero-Knowledge Proofs for the Formal Verification of Neural Networks (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.