Research map / Adversarial robustness and security / Certified and empirical robustness
The Lipschitz-Variance-Margin Tradeoff for Enhanced Randomized Smoothing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LVM-RS, a randomized smoothing procedure that jointly exploits the Monte-Carlo variance, Lipschitz constant and margin via r-simplex generalized sparsemax/softmax/hardmax maps with temperature, Empirical Bernstein risk control, and tighter smoothed-classifier Lipschitz bounds.
Problem
Randomized smoothing certification relies on Monte-Carlo estimates whose variance and risk correction (Clopper-Pearson/Hoeffding) shrink the certified radius; the interplay between base classifier Lipschitz constant, margin and sampling variance is not exploited.
Approach
Use the Gaussian-Poincaré inequality to link base-classifier Lipschitz constant to MC variance, Empirical Bernstein bounds to control risk using observed variance, and an r-simplex generalized sparsemax/softmax/hardmax map with temperature selected per input to trade off variance against margin; also derive tighter Lipschitz bounds for the smoothed classifier.
Key results
- On CIFAR-10 with n=1e5 samples, LVM-RS best certified accuracy at epsilon 0.0 is 88.49 vs 86.72 for the baseline (Carlini et al.), and 32.35 vs 29.91 at epsilon 1.0.
- On ImageNet (n=1e4), LVM-RS certified accuracy at epsilon 2 is 27.43 vs 25.53 for the baseline, and 53.85 vs 51.55 at epsilon 1.0.
- On CIFAR-10 with a 5-Lipschitz Sandwich Small backbone, the new Lipschitz bound yields certified accuracy 52.56 at epsilon 0.14 vs 47.9 for standard randomized smoothing.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the new Lipschitz bound for the smoothed classifier instead of the conventional RS or deterministic Lipschitz bound | Randomized smoothing with the Sandwich Small backbone · CIFAR-10 | certified accuracy: 47.9 -> 52.56 (ε=0.14); 31.99 -> 46.17 (ε=0.19); 28.17 -> 39.09 (ε=0.25); 27.86 -> 35.08 (ε=0.28); 6.42 -> 21.9 (ε=0.42) | Helped |
| Select a simplex map and temperature using the variance-margin tradeoff, with empirical Bernstein risk control | Randomized smoothing with a pretrained vision transformer and denoiser · CIFAR-10 | certified accuracy: 86.72 -> 88.49 (ε=0.0); 74.41 -> 76.21 (ε=0.25); 58.25 -> 60.22 (ε=0.5); 40.96 -> 43.76 (ε=0.75); 29.91 -> 32.35 (ε=1.0) | Helped |
| Select a simplex map and temperature using the variance-margin tradeoff, with empirical Bernstein risk control | Randomized smoothing with a pretrained vision transformer and denoiser · ImageNet | certified accuracy: 79.88 -> 80.66 (ε=0.0); 69.57 -> 69.84 (ε=0.5); 51.55 -> 53.85 (ε=1.0); 25.53 -> 27.43 (ε=2); 14.01 -> 14.31 (ε=3) | Helped |
| Select a simplex map and temperature using the variance-margin tradeoff, with empirical Bernstein risk control | Randomized smoothing with a pretrained vision transformer and denoiser · ImageNet, ε=1.5 | certified accuracy: 36.04 -> 36.04 | No clear effect |
Limitations
- Paper states future work to integrate with margin maximization and explore the choice of simplex mass r.
- Lipschitz constant of the base classifier is hard (NP-hard) to compute for common networks, so bounds may overestimate empirical variance.
- Compute is reported as V100 GPU, number of GPUs not stated.
Experimental setup
Datasets: CIFAR-10 ImageNet
Benchmarks: CIFAR-10 ImageNet
Metrics: certified accuracy average time (s) certified radius
Baselines: Randomized smoothing (Cohen et al.) Lipschitz deterministic certificate Carlini et al. ViT denoiser smoothing
Builds on: randomized smoothing Clopper-Pearson certification (Cohen et al.) Sandwich layers Lipschitz networks sparsemax Diffusion denoised smoothing (Carlini et al.)
Closest papers by meaning
- Certifying Confidence via Randomized Smoothing (NeurIPS 2020 · similarity 0.92)
- Towards Large Certified Radius in Randomized Smoothing Using Quasiconcave Optimization (AAAI 2024 · similarity 0.90)
- Certified Robustness for Deep Equilibrium Models via Serialized Random Smoothing (NeurIPS 2024 · similarity 0.90)
- Incremental Randomized Smoothing Certification (ICLR 2024 · similarity 0.89)
- Average Certified Radius is a Poor Metric for Randomized Smoothing (ICML 2025 · similarity 0.88)
- Mitigating the Curse of Dimensionality for Certified Robustness via Dual Randomized Smoothing (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.