Research map / Adversarial robustness and security / Certified and empirical robustness
Certified Robustness for Deep Equilibrium Models via Serialized Random Smoothing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Applies randomized smoothing to certify deep equilibrium models (MDEQ) and proposes Serialized Randomized Smoothing, which warm-starts each noisy sample's fixed-point solve from the previous one, plus a correlation-eliminated radius estimate, yielding up to 7x certification speedup on CIFAR-10 and ImageNet.
Problem
Certifying the l2 robustness of deep equilibrium models is expensive: randomized smoothing needs many Monte Carlo samples, each requiring a fixed-point solve, and prior deterministic certifiers (IBP, Lipschitz bounds) are conservative and tied to specific DEQ architectures and do not scale to ImageN
Approach
Warm-start the fixed-point solver for each noisy sample from the previous sample's equilibrium so only a few solver steps are needed, and to preserve the randomized-smoothing guarantee, estimate an upper bound on the probability of prediction mismatch with a small hypothesis-tested subsample and discard those samples when computing the certified radius.
Key results
- SRS-MDEQ-1A achieves about 7x speedup over MDEQ-30A on CIFAR-10 large (1.79s vs 12.89s) with certified accuracy at radius 0.5 of 45% vs 45%.
- SRS-MDEQ-1B on ImageNet (MDEQ-SMALL, sigma=1.0) runs in 15.21s versus 88.33s for MDEQ-14B (6x), with certified accuracy at radius 1.0 of 32% vs 33%.
- Small SRS-MDEQ-3A on CIFAR-10 is about 7x faster than MDEQ-30A... with 1% accuracy drop as stated in text, while Table reports 3x (1.14s vs 3.08s) for the small model.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use serialized smoothing with 1 Anderson step instead of standard MDEQ-30A | MDEQ-LARGE · CIFAR-10 | time per image: 12.89 -> 1.79 | Helped |
| Use serialized smoothing with 3 Anderson steps instead of standard MDEQ-30A | MDEQ-SMALL · CIFAR-10 | time per image: 3.08 -> 1.14 | Helped |
| Use serialized smoothing with 3 Broyden steps instead of standard MDEQ-14B | MDEQ-SMALL · ImageNet | time per image: 88.33 -> 27.48 | Helped |
| Use SRS-MDEQ-3A instead of ResNet-110 | CIFAR-10 image classification · CIFAR-10, radius 0.5 | certified accuracy: 41% -> 45% | Helped |
| Add Jacobian regularization (ablation) | MDEQ-30A · CIFAR-10, radius 0.5 | certified accuracy: 38% -> 38% | No clear effect |
| Add Jacobian regularization (ablation) | SRS-MDEQ-3A · CIFAR-10, radius 0.5 | certified accuracy: 38% -> 38% | No clear effect |
| Increase SRS solver steps from 3 to 5 (ablation) | SRS-MDEQ · CIFAR-10, MDEQ-SMALL, radius 0.5 | certified accuracy: 38% -> 38% | No clear effect |
| Replace the Anderson solver with the naive solver (ablation) | MDEQ-LARGE-30 · CIFAR-10, radius 0.5 | certified accuracy: 45% -> 42% | Hurt |
| Use SmoothAdv training instead of standard training (ablation) | MDEQ-30A · CIFAR-10, MDEQ-SMALL, radius 0.5 | certified accuracy: 38% -> 43% | Helped |
Surprising findings
- Reported speedups in the text (7x for the small model) do not match the table values (3x for SRS-MDEQ-3A small), indicating internal inconsistency.
- Jacobian regularization has little effect on final certified accuracy with enough fixed-point iterations, per the appendix ablation.
Limitations
- Certification is probabilistic (failure rate alpha), not deterministic, so results are not fully comparable to deterministic certifiers.
- Method is specific to DEQ-style fixed-point solvers and the paper notes it cannot be directly applied to other architectures.
- Experiments are limited to MDEQ on image classification with a single A100 GPU.
Experimental setup
Datasets: CIFAR-10 ImageNet
Benchmarks: CIFAR-10 ImageNet
Metrics: approximate certified accuracy average certified radius running time per image relative radius difference
Baselines: standard randomized smoothing on MDEQ-30A SLL LBEN ResNet-110 IBP
Builds on: randomized smoothing MDEQ SmoothAdv
Closest papers by meaning
- Incremental Randomized Smoothing Certification (ICLR 2024 · similarity 0.91)
- Certified Adversarial Robustness via Randomized $\alpha$-Smoothing for Regression Models (NeurIPS 2024 · similarity 0.90)
- The Lipschitz-Variance-Margin Tradeoff for Enhanced Randomized Smoothing (ICLR 2024 · similarity 0.90)
- Halt Fast! Early Stopping for Certified Robustness (NeurIPS 2026 · similarity 0.88)
- Robust Representation Consistency Model via Contrastive Denoising (ICLR 2025 · similarity 0.88)
- Adaptive Randomized Smoothing: Certified Adversarial Robustness for Multi-Step Defences (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.