Research map / Architectures and efficiency / Attention and transformers
Certified Patch Robustness via Smoothed Vision Transformers
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace a similarly sized ResNet base classifier with a ViT in derandomized smoothing | Smoothed image classifiers · ImageNet, 2% pixels | certified accuracy: 18.3 -> 31.6 | Helped |
| Replace a ResNet base classifier with a ViT in derandomized smoothing | Smoothed image classifiers · CIFAR-10, 4×4 patch | certified accuracy: 59.0 -> 63.8 | Helped |
| Increase test-time column ablation width from b = 19 to b = 37 (ablation) | ViT-B smoothed classifier · ImageNet | standard accuracy: 69.3 -> 73.2 | Helped |
| Increase test-time column ablation width from b = 19 to b = 37 (ablation) | ViT-B smoothed classifier · ImageNet, 2% pixels | certified accuracy: 38.3 -> 38.2 | No clear effect |
| Increase test-time column ablation width from b = 19 to b = 25 (ablation) | ViT-B smoothed classifier · ImageNet, 2% pixels | certified accuracy: 38.3 -> 38.8 | Helped |
| Use stride s = 10 for column ablations (ablation) | ViT-B smoothed classifier · ImageNet, 2% pixels | certified accuracy: 38.3 -> 36.9 | Hurt |
| Use stride s = 10 for column ablations (ablation) | ViT-B smoothed classifier · ImageNet | inference time: 31.5 -> 3.2 sec | Helped |
| Increase training-time column ablation width for ViT-S to b = 37 instead of b = 19 (ablation) | ViT-S smoothed classifier · ImageNet | certified accuracy: 1.3% | Helped |
| Increase training-time column ablation width for ResNet-50 to b = 37 (ablation) | ResNet-50 smoothed classifier · ImageNet | certified accuracy: 18% to 27% | Helped |
| Drop tokens corresponding to fully masked image regions (ablation) | Vision Transformer base classifier · ImageNet and CIFAR-10 image ablations | ablation accuracy | No clear effect |
| Drop fully masked tokens during smoothing (ablation) | ViT smoothed classifier · ImageNet, b = 19 | inference speedup over smoothed ResNets: 5.85x; 10.68x; 22.04x | Helped |
| Use block ablations instead of column ablations (ablation) | ViT-B smoothed classifier · ImageNet | certified accuracy: up to 5% | Hurt |
Closest papers by meaning
- Improving Robustness of Vision Transformers by Reducing Sensitivity To Patch Corruptions (CVPR 2023 · similarity 0.90)
- Towards Practical Certifiable Patch Defense With Vision Transformer (CVPR 2022 · similarity 0.90)
- Mitigating the Curse of Dimensionality for Certified Robustness via Dual Randomized Smoothing (ICLR 2024 · similarity 0.85)
- An Image is Worth More Than 16x16 Patches: Exploring Transformers on Individual Pixels (ICLR 2025 · similarity 0.84)
- Improving Interpretation Faithfulness for Vision Transformers (ICML 2024 · similarity 0.84)
- Pixel-level Certified Explanations via Randomized Smoothing (ICML 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.