Research map / Adversarial robustness and security / Certified and empirical robustness
A Recipe for Improved Certifiable Robustness
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Synthesizes and extends Lipschitz-based certified training (LiResNet, GloRo) by adding Cholesky-orthogonalized residual dense layers, orthogonal dense-layer control, and filtered EDM-generated data augmentation, reaching new deterministic verified robust accuracy records on CIFAR-10/100, Tiny-ImageNet and ImageNet.
Problem
Lipschitz-based certified training underfits because tight Lipschitz control limits effective model capacity, leaving a performance gap versus randomized smoothing for deterministic certification.
Approach
Add large residual dense layers whose weights are orthogonalized via Cholesky decomposition at the end of a LiResNet neck (keeping GloRo regularization for convolutions), and augment training with higher-quality EDM-generated images filtered by a pretrained classifier's confidence on pseudo-labels.
Key results
- GloRo CHORD LiResNet reaches 78.1% VRA at epsilon=36/255 on CIFAR-10 (vs 70.0% for GloRo LiResNet +DDPM) and 50.1% on CIFAR-100 (vs 41.5%).
- On ImageNet, VRA at epsilon=36/255 improves from 35.0% to 38.3% (3.3 points) over GloRo LiResNet.
- Ablation: Cholesky-orthogonalized residual dense layers match matrix exponential best on CIFAR-10 (78.1 vs 78.1 VRA) and train faster than matrix exponential (32.4 s vs 51.2 s per epoch).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add 8 Spatial-MLP blocks after the last convolution layers (ablation) | LiResNet L12W512 · CIFAR-10 and CIFAR-100, ε=36/255 | VRA (%): 70.0 -> 70.2; 41.5 -> 41.2 | Mixed |
| insert 1 Spatial-MLP block after every convolution layer (ablation) | LiResNet L12W512 · CIFAR-10 and CIFAR-100, ε=36/255 | VRA (%): 70.0 -> 70.3; 41.5 -> 41.7 | Helped |
| add 8 dense layers before the classification head (ablation) | LiResNet L12W512 · CIFAR-10 and CIFAR-100, ε=36/255 | VRA (%): 70.0 -> 73.4; 41.5 -> 43.6 | Helped |
| increase the dense-layer count from 8 to 16 (ablation) | LiResNet L12W512 · CIFAR-10 and CIFAR-100, ε=36/255 | VRA (%): 73.4 -> 73.7; 43.6 -> 44.0 | Helped |
| use Cholesky-based orthogonalization in the neck (ablation) | LiResNet with 8 dense layers · CIFAR-10 and CIFAR-100, ε=36/255 | VRA (%): 73.4 -> 74.5; 43.6 -> 44.9 | Helped |
| replace the 8 dense layers with CHORD layers (ablation) | LiResNet with Cholesky-orthogonalized neck · CIFAR-10 and CIFAR-100, ε=36/255 | VRA (%): 74.5 -> 75.6; 44.9 -> 46.5 | Helped |
| adopt the improved generated-data augmentation pipeline (ablation) | LiResNet with CHORD layers · CIFAR-10 and CIFAR-100, ε=36/255 | VRA (%): 75.6 -> 78.1; 46.5 -> 50.1 | Helped |
| switch the sample generator from IDDPM to EDM (ablation) | GloRo CHORD LiResNet with unfiltered generated samples and a 1:1 real/generated batch ratio · CIFAR-10 and CIFAR-100, ε=36/255 | VRA (%): 75.6 -> 76.1; 46.5 -> 47.3 | Helped |
| filter generated samples using classification confidence (ablation) | GloRo CHORD LiResNet with EDM and a 1:1 real/generated batch ratio · CIFAR-10 and CIFAR-100, ε=36/255 | VRA (%): 76.1 -> 76.5; 47.3 -> 48.5 | Helped |
| increase the generated-sample share to a 1:3 real/generated batch ratio (ablation) | GloRo CHORD LiResNet with filtered EDM samples · CIFAR-10 and CIFAR-100, ε=36/255 | VRA (%): 76.5 -> 78.1; 48.5 -> 50.1 | Helped |
| train using only generated samples (0:1 real/generated ratio) (ablation) | GloRo CHORD LiResNet with filtered EDM samples · CIFAR-10 and CIFAR-100, ε=36/255 | VRA (%): 78.1 -> 75.4; 50.1 -> 47.6 | Hurt |
| replace regularized dense layers with Cayley-orthogonalized dense layers (ablation) | Modified LiResNet dense layers · CIFAR-10, ε=36/255 | VRA (%): 77.0 -> 77.2 | Helped |
Surprising findings
- Training only on generated samples (real/generated ratio 0:1) overfits and reduces VRA, while mixing in generated data improves it, which the authors attribute to generated samples being easier to classify rather than higher quality.
- Spatial-MLP blocks gave negligible gains (within about 0.3 points) for certified training despite comparable performance to transformers in standard training.
Limitations
- Gains are largely from combining many prior techniques and tuning, so individual novelty is limited; the authors frame it as a synthesis.
- Convolutional layers still use GloRo regularization since orthogonalizing convolutions via FFTs is expensive; the paper does not orthogonalize them.
- ImageNet results use only one comparison point (GloRo LiResNet) and generated data from guided diffusion, with no broader ImageNet comparison.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 Tiny-ImageNet ImageNet
Benchmarks: CIFAR-10 CIFAR-100 Tiny-ImageNet ImageNet
Metrics: clean accuracy verified robust accuracy (VRA) AutoAttack robust accuracy
Baselines: GloRo Local-Lip-B Cayley Large SOC CPL AOL SLL Sandwich Randomized smoothing variants (RS, SmoothAdv, MACER, DRT, DDS, etc.)
Builds on: LiResNet GloRo training EDM Sandwich layer Cayley transform
Closest papers by meaning
- Enhancing Certified Robustness via Block Reflector Orthogonal Layers and Logit Annealing Loss (ICML 2025 · similarity 0.92)
- Efficient Robust Conformal Prediction via Lipschitz-Bounded Networks (ICML 2025 · similarity 0.91)
- 1-Lipschitz Layers Compared: Memory Speed and Certifiable Robustness (CVPR 2024 · similarity 0.91)
- ECLipsE: Efficient Compositional Lipschitz Constant Estimation for Deep Neural Networks (NeurIPS 2024 · similarity 0.90)
- Learning Better Certified Models from Empirically-Robust Teachers (NeurIPS 2026 · similarity 0.90)
- Certified Robustness Under Bounded Levenshtein Distance (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.