Research map / Adversarial robustness and security / Certified and empirical robustness
Expressive Losses for Verified Robustness via Convex Combinations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes expressive losses (convex combinations of adversarial and IBP verified losses) for verified training, showing simple instances (CC-IBP, MTL-IBP, Exp-IBP) reach state-of-the-art accuracy-robustness trade-offs on CIFAR-10, TinyImageNet and ImageNet64.
Problem
Verified adversarial training suffers a trade-off between standard accuracy and verified robustness; existing methods (SABR, CROWN-IBP, COLT, IBP-R) combine attacks and over-approximations heuristically, and it is unclear what property of the loss drives performance.
Approach
Define "expressive" losses as a family parameterized by a single over-approximation coefficient alpha that interpolates monotonically from the adversarial loss (alpha=0) to the IBP verified loss (alpha=1); instantiate with simple convex combinations (CC-IBP on logit differences, MTL-IBP on losses, Exp-IBP on log-losses) and show that tuning alpha yields SOTA, while tighter worst-case approximation
Key results
- On TinyImageNet at epsilon=1/255, Exp-IBP reaches 38.71% standard and 26.18% verified accuracy, improving standard accuracy by at least 8.58% over prior literature results.
- On ImageNet64 at epsilon=1/255, Exp-IBP achieves 22.73% standard and 13.30% verified robust accuracy, versus 9.54% for the prior best (SortNet).
- On CIFAR-10 at epsilon=2/255, CC-IBP reaches 80.09% standard and 63.78% verified robust accuracy, beating the literature results on the trade-off except SortNet at 8/255.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a tuned expressive loss (CC-IBP, MTL-IBP, or Exp-IBP) instead of previously reported methods | verified robust training · TinyImageNet ε=1/255 | standard and verified robust accuracy: +8.58% standard accuracy; +3.93% verified robust accuracy | Helped |
| Use a tuned expressive loss (CC-IBP, MTL-IBP, or Exp-IBP) instead of previously reported methods | verified robust training · ImageNet64 ε=1/255 | standard and verified robust accuracy: +3.39% standard accuracy; +2.33% verified robust accuracy | Helped |
| Tune MTL-IBP's scalarization instead of using AdvIBP | verified robust training · CIFAR-10, TinyImageNet, and ImageNet64 | robustness-accuracy performance | Helped |
| Use MTL-IBP with α=0.5 instead of the COLT-like stage-wise procedure | CIFAR-10 ε=8/255 · CIFAR-10 ε=8/255 | IBP verified robust accuracy: 26.22% -> 34.61% | Helped |
| Increase α | CC-IBP, MTL-IBP, and Exp-IBP · CIFAR-10 | standard accuracy | Hurt |
| Increase α | CC-IBP, MTL-IBP, and Exp-IBP · CIFAR-10, generally | IBP verified accuracy | Helped |
| Increase α | CC-IBP, MTL-IBP, and SABR · CIFAR-10 under tighter verification | verified robust accuracy | Mixed |
| Use a more accurate approximation of the branch-and-bound loss | expressive-loss training · CIFAR-10 ε=2/255 and ε=8/255 | accuracy–robustness trade-off | Mixed |
| Use a more accurate worst-case loss approximation | expressive-loss training · CIFAR-10 ε=2/255 | verified robustness and accuracy trade-off | No clear effect |
Surprising findings
- Better approximations of the branch-and-bound worst-case loss do not necessarily yield better robustness-accuracy trade-offs; over-approximations performed better at epsilon=2/255 and under-approximations at epsilon=8/255.
- Exp-IBP is extremely sensitive to alpha on CIFAR-10 at epsilon=2/255, with good trade-offs only in a small neighborhood of alpha=0.1.
Limitations
- Branch-and-bound verification on ImageNet64 was only attempted for alpha >= 5e-2, which may skew comparisons in favor of Exp-IBP.
- Table results tune directly on the evaluation sets, following common practice, so reported numbers may be optimistic.
- Results are limited to vision benchmarks with l_inf perturbations and a 7-layer architecture (plus a 2x wider network for some baselines).
Experimental setup
Datasets: CIFAR-10 TinyImageNet ImageNet64 MNIST
Benchmarks: CIFAR-10 TinyImageNet ImageNet64
Metrics: standard accuracy verified robust accuracy adversarial accuracy
Baselines: SABR STAPS SortNet IBP-R IBP AdvIBP CROWN-IBP COLT
Builds on: SABR CROWN-IBP IBP COLT IBP-R AdvIBP OVAL auto_LiRPA
Closest papers by meaning
- Theoretically Principled Trade-off between Robustness and Accuracy (ICML 2019 · similarity 0.91)
- Towards Better Understanding of Training Certifiably Robust Models against Adversarial Examples (NeurIPS 2021 · similarity 0.90)
- Semidefinite relaxations for certifying robustness to adversarial examples (NeurIPS 2018 · similarity 0.90)
- Adversarial Training and Provable Robustness: A Tale of Two Objectives (AAAI 2021 · similarity 0.90)
- Quadratic Upper Bound for Boosting Robustness (ICML 2025 · similarity 0.89)
- Learning Better Certified Models from Empirically-Robust Teachers (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.