AI Paper Atlas

Research map / Adversarial robustness and security / Certified and empirical robustness

Low Curvature Activations Reduce Overfitting in Adversarial Training

Vasu Singla, Sahil Singla, Soheil Feizi, David Jacobs

ICCV 2021 · Adversarial robustness and security › Certified and empirical robustness · approach: Robust overfitting and regularization · 39 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use SiLU instead of LiSHT (ablation)adversarially trained ResNet-18 · CIFAR-10robust generalization gap: 45.06 -> 7.24Improved
use SiLU instead of ReLUadversarially trained ResNet-18 · CIFAR-100robust test accuracy (final): 18.81 -> 24.29Improved
use SiLU instead of ReLUadversarially trained ResNet-18 · CIFAR-10robust generalization gap: 33.21 -> 7.24Improved
use SiLU instead of LiSHT (ablation)adversarially trained ResNet-18 · CIFAR-10robust overfitting gap (best vs final): 3.1% -> 0.5%Improved
use LiSHT (smooth, high curvature) instead of ReLUadversarially trained ResNet-18 · CIFAR-10robust test accuracy (final): 49.25 -> 47.21Hurt
lower PSwish beta from 10 to 0.5 (lower curvature) (ablation)PSwish ResNet-18 · CIFAR-10robust generalization gap: 38.57 -> 1.76Improved
lower PSwish beta from 10 to 0.5 (lower curvature) (ablation)PSwish ResNet-18 · CIFAR-10robust test accuracy (final): 50.63 -> 45.24Hurt
lower PSwish beta from 10 to 0.5 (lower curvature) (ablation)PSwish ResNet-18 · CIFAR-10standard generalization gap: 16.13 -> 1.82Improved
raise LeakyReLU slope k from -0.2 to 0.5 (lower approximate curvature) (ablation)LeakyReLU ResNet-18 · CIFAR-10robust generalization gap: 37.73 -> 4.03Improved
raise LeakyReLU slope k from -0.2 to 0.5 (lower approximate curvature) (ablation)LeakyReLU ResNet-18 · CIFAR-10standard generalization gap: 16.46 -> 3.43Improved
use PSwish beta=10 instead of ReLU with early stoppingadversarially trained ResNet-18 · CIFAR-10robust and standard accuracy: +0.33% robust, +1.24% standardImproved
use SiLU instead of ReLU in wide ResNetsadversarially trained WideResNet · width sweeprobust test error vs width: width-factor 4 equals ReLU width-factor 15Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy