Research map / Adversarial robustness and security / Certified and empirical robustness

Improving robustness to corruptions with multiplicative weight perturbations

NeurIPS 2024 · Adversarial robustness and security › Certified and empirical robustness · approach: Adversarial training recipes · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DAMP, a training method that injects random multiplicative Gaussian weight perturbations at the same cost as SGD to improve corruption robustness, and reinterprets ASAM as training under adversarial multiplicative weight perturbations.

Problem

Deep neural networks degrade on corrupted images; augmenting training with specific corruptions helps those corruptions but can hurt clean accuracy and robustness to other corruptions.

Approach

Input corruptions can be simulated by multiplicative perturbations of weights (z = (w∘ξ)ᵀx with ξ = 1 + ε/x); DAMP trains under random Gaussian multiplicative weight noise split across sub-batches so the cost matches SGD, and ASAM is shown to correspond to adversarial multiplicative perturbations.

Key results

• DAMP trains ViT-S/16 from scratch on ImageNet with basic augmentation to 23.75% clean top-1 error at 200 epochs, comparable to ResNet50 without advanced augmentation.
• On ResNet50/ImageNet, DAMP attains the lowest average corrupted error (74.2%) among Dropout, SAM, and ASAM at half the training cost of SAM/ASAM.
• With MixUp and RandAugment, DAMP-trained ViT-B/16 reaches 59.19% average corrupted error versus 61.61% for SAM-trained ViT-S/16 under comparable training time.

Measured effects reported in the paper

Surprising findings

• DAMP improves clean accuracy over Dropout on CIFAR-100 and TinyImageNet, and over SGD on PreActResNet18/TinyImageNet.
• DAMP outperforms SAM on corrupted images in most cases despite SAM's double training cost.

Limitations

• The bound/proof assumes a simple feedforward network and does not cover normalization layers or attention.
• Only Gaussian multiplicative perturbations were explored.
• Omits the Hessian term in the ASAM derivation, so the ASAM connection is approximate.

Experimental setup

Datasets: CIFAR-10 CIFAR-100 TinyImageNet ImageNet

Benchmarks: CIFAR-10-C CIFAR-100-C TinyImageNet-C ImageNet-C ImageNet-Cbar ImageNet-A ImageNet-D ImageNet-Sketch ImageNet-Cartoon ImageNet-Drawing ImageNet-Hard FGSM adversarial examples

Metrics: top-1 error corruption error (CE)

Baselines: SGD Dropout SAM ASAM AugMix Patch Gaussian AdamW DAAP

Builds on: SAM ASAM Dropout AugMix

Closest papers by meaning

• Benchmarking Neural Network Robustness to Common Corruptions and Perturbations (ICLR 2019 · similarity 0.88)
• Feature Collapse Under Corruption: An Entropy Perspective on Robust Neural Networks (ICML 2026 · similarity 0.88)
• Learning perturbation sets for robust machine learning (ICLR 2021 · similarity 0.87)
• Verification of Neural Networks Against Convolutional Perturbations via Parameterised Kernels (AAAI 2025 · similarity 0.86)
• Slight Corruption in Pre-training Data Makes Better Diffusion Models (NeurIPS 2024 · similarity 0.85)
• Dynamic Sparse Training versus Dense Training: The Unexpected Winner in Image Corruption Robustness (ICLR 2025 · similarity 0.85)