Research map / Adversarial robustness and security / Certified and empirical robustness

Quadratic Upper Bound for Boosting Robustness

ICML 2025 · Adversarial robustness and security › Certified and empirical robustness · approach: Provable robust learning guarantees · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Derives a quadratic upper bound on the adversarial training cross-entropy loss and uses it (static or decreasing weighting) as a drop-in replacement loss for fast adversarial training methods to improve robustness and flatten the loss landscape.

Problem

Fast adversarial training (FAT) with single-step attacks explores the adversarial space too coarsely, leading to degraded robustness and catastrophic overfitting.

Approach

Derive a quadratic upper bound on the AT loss using Taylor expansion and the convexity of cross-entropy in logits (Hessian norm bounded by 1/2), yielding a loss with a clean-loss term, a gradient-alignment term approximated by delta^T grad_x L, and a logit-change penalty; replace the AT loss with this bound, optionally annealing toward AT loss over training.

Key results

• On CIFAR-10 ResNet18, adding QUB-decreasing to Free-AT raises PGD10 robustness from 45.32% to 45.58% and AA from 41.38% to 41.60%; QUB-static to FGSM-CKPT raises AA from 37.22% to 41.53%.
• QUB-static to FGSM-CKPT raises PGD10 robustness from 41.19% to 45.41%, while SA drops from 90.02% to 87.63%.
• Adding QUB reduces average dominant Hessian eigenvalue of input-loss, and increases adversarial sparsity across epsilons for all tested methods.

Measured effects reported in the paper

Surprising findings

• Combining QUB with FGSM-RS fails to improve and can degrade robustness, attributed to FGSM-RS's poor exploration.
• QUB-static gives better robustness but substantially lower standard accuracy; on CIFAR-100 and Tiny ImageNet, QUB-static was dropped for excessive SA loss.

Limitations

• Bound is derived via a local Taylor approximation and is effective mainly for small perturbations; gap to AT loss grows with epsilon, and robustness gains diminish at larger epsilon.
• QUB-static substantially reduces standard accuracy; QUB-decreasing needed to balance, and gains are inconsistent (e.g., FGSM-RS degraded).

Experimental setup

Datasets: CIFAR-10 CIFAR-100 Tiny ImageNet

Benchmarks: CIFAR-10 CIFAR-100 Tiny ImageNet AutoAttack

Metrics: standard accuracy robust accuracy training time dominant Hessian eigenvalue adversarial sparsity

Baselines: Free-AT FGSM-RS FGSM-CKPT FGSM-GA FGSM-PGI N-FGSM FGSM-UAP ELLE-A PGD-AT TRADES NuAT GAT

Builds on: PGD adversarial training FGSM-RS Free-AT N-FGSM TRADES

Closest papers by meaning

• Expressive Losses for Verified Robustness via Convex Combinations (ICLR 2024 · similarity 0.89)
• Certified Adversarial Robustness with Additive Noise (NeurIPS 2019 · similarity 0.89)
• Attacks Which Do Not Kill Training Make Adversarial Learning Stronger (ICML 2020 · similarity 0.89)
• Bag of Tricks for Adversarial Training (ICLR 2021 · similarity 0.89)
• Theoretically Principled Trade-off between Robustness and Accuracy (ICML 2019 · similarity 0.88)
• On the Scalability of Certified Adversarial Robustness with Generated Data (NeurIPS 2024 · similarity 0.88)