Research map / Adversarial robustness and security / Certified and empirical robustness
Treatment of Statistical Estimation Problems in Randomized Smoothing for Adversarial Robustness
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Replaces fixed-sample Clopper-Pearson estimation in randomized smoothing with an optimal randomized Clopper-Pearson interval and anytime confidence sequences (union-bound and betting-based) that reduce the number of classifier forward passes needed for certification.
Problem
Randomized smoothing certification requires many (~10^5) classifier forward passes per input, and the standard fixed-n Clopper-Pearson estimation is conservative and wastes samples when a decision could be made early.
Approach
Introduces a randomized Clopper-Pearson interval that is uniformly more powerful than any other binomial interval at the same coverage, and uses confidence sequences (union-bound stitching over doubling times and a Krichevsky-Trofimov coin-betting construction) so that samples are drawn adaptively until a robustness decision is reached, with error control over all times simultaneously.
Key results
- On CIFAR-10 (l2, sigma=0.12) the betting confidence sequence needs roughly 531 samples at r=0.5 versus 1976 for the adaptive baseline of prior work.
- Union-bound and betting confidence sequences reduce the average samples needed to decide robustness by roughly 2-3x across radii compared to the adaptive baseline.
- The randomized Clopper-Pearson interval has exact 1-alpha coverage and is shown to be optimal among binomial confidence intervals.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use betting confidence sequences instead of Adaptive | Sequential robustness certification · CIFAR-10 ℓ2, r=0.5 | samples: 1976 ± 41 -> 531 ± 157 | Helped |
| Use union-bound confidence sequences instead of Adaptive | Sequential robustness certification · CIFAR-10 ℓ2, r=1.25 | samples: 3593±574 -> 2557 ± 234 | Helped |
| Use betting confidence sequences instead of Adaptive | Sequential robustness certification · CIFAR-10 ℓ2, r=2 | samples: 4623 ±47 -> 2130 ± 339 | Helped |
| Use betting confidence sequences instead of Adaptive | Sequential robustness certification · CIFAR-10 ℓ2, σ=1 retrained, r=0.5 | samples: 4150 ± 523 -> 2206 ± 150 | Helped |
| Use betting confidence sequences instead of Adaptive | Sequential robustness certification · CIFAR-10 ℓ1, r=0.5 | samples: 423 ± 38 -> 138 ± 28 | Helped |
| Use confidence sequences instead of Horvath | Sequential Bernoulli estimation · CIFAR-10 ℓ2, confidence width ε=0.01 | samples: 768~560 -> 197~628 | Helped |
Surprising findings
- Standard Clopper-Pearson intervals are conservative, causing sharp drops at the end of robustness curves for large p_A.
- Under the standard fixed n=10^5 setting, the implicit type-2 error can be large for samples far from the threshold, so fixed-sample certification is often not the most practically relevant setting.
Limitations
- Betting confidence sequence is not asymptotically optimal (width scales with log t rather than log log t).
- Experiments are limited to CIFAR-10 with Gaussian and uniform smoothing; no ImageNet or non-image modalities evaluated.
- Gains are reported on a sequential task rather than full robustness curves, so the comparison of robustness curves is vacuous in the adaptive setting.
Experimental setup
Datasets: CIFAR-10
Benchmarks: CIFAR-10
Metrics: average number of samples wall-clock time certified radius
Baselines: Clopper-Pearson interval Adaptive (Sequential certification) Horvath et al. sample allocation
Builds on: randomized smoothing Clopper-Pearson interval Ville's inequality Krichevsky-Trofimov estimator
Closest papers by meaning
- Average Certified Radius is a Poor Metric for Randomized Smoothing (ICML 2025 · similarity 0.90)
- Certified Adversarial Robustness via Randomized $\alpha$-Smoothing for Regression Models (NeurIPS 2024 · similarity 0.90)
- Effects of Exponential Gaussian Distribution on (Double Sampling) Randomized Smoothing (ICML 2024 · similarity 0.90)
- Adversarial Robustness via Deformable Convolution with Stochasticity (ICML 2025 · similarity 0.90)
- Adaptive Randomized Smoothing: Certified Adversarial Robustness for Multi-Step Defences (NeurIPS 2024 · similarity 0.89)
- Provably Cost-Sensitive Adversarial Defense via Randomized Smoothing (ICML 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.