Research map / Adversarial robustness and security / Certified and empirical robustness

Democratic Training Against Universal Adversarial Perturbations

ICLR 2025 · Adversarial robustness and security › Certified and empirical robustness · approach: Adversarial training refinements · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Democratic Training, an entropy-guided adversarial fine-tuning defense that repairs CNNs against targeted universal adversarial perturbations by training on PGD-generated low-entropy samples from a small clean subset.

Problem

Neural networks are vulnerable to universal adversarial perturbations (UAPs), input-agnostic perturbations that fool a model on many clean inputs; existing defenses need many pre-computed UAPs or architecture changes.

Approach

Observes that targeted UAPs drop layer-wise activation entropy in deep layers (features dominated by few units), and fine-tunes the model on low-entropy samples generated from clean inputs via PGD on a layer-wise entropy loss, without generating UAPs or changing architecture.

Key results

• Across 7 models, average targeted UAP attack success rate drops from 79.4% to 2.8% and adversarial accuracy rises from 12.6% to 69.3%, with clean accuracy down about 2%.
• Against sPGD, LaVAN, GAP and SGA UAPs, post-defense attack success rates are below 0.9%, 1.4%, 2.5% and 2.9% respectively.
• Compared with SFR, CFN, FNS and DensePure, Democratic Training gives comparable or better robustness without architecture changes; DensePure drops clean accuracy by 12.1%.

Measured effects reported in the paper

Surprising findings

• Non-targeted UAPs show no clear entropy separation, and the defense reduces their attack success rate only to about 30% on average.
• Adaptive secondary DF-UAP attacks on the repaired models still reach 23.6% average success, though far below the pre-defense rate.

Limitations

• Evaluated only on image classification with CNN architectures; transformers and non-vision tasks left to future work.
• Non-targeted UAP defense is much weaker (about 30% residual attack success).
• Defense is evaluated against UAPs with the l_inf budget used in training; larger epsilon (15/255) raises attack success to 11.5%.

Experimental setup

Datasets: ImageNet ASL Alphabet Caltech101 EuroSAT CIFAR-10

Benchmarks: ImageNet ASL Alphabet Caltech101 EuroSAT CIFAR-10

Metrics: attack success rate adversarial accuracy clean accuracy change execution time

Baselines: PGD adversarial training (targeted) PGD adversarial training (non-targeted) finetuning with known UAP TRADES SFR CFN FNS DensePure

Builds on: DF-UAP PGD adversarial training TRADES selective feature regeneration

Closest papers by meaning

• Learning Universal Adversarial Perturbation by Adversarial Example (AAAI 2022 · similarity 0.91)
• Data-Free Universal Attack by Exploiting the Intrinsic Vulnerability of Deep Models (AAAI 2025 · similarity 0.90)
• Improving Generalization of Universal Adversarial Perturbation via Dynamic Maximin Optimization (AAAI 2025 · similarity 0.89)
• Generate Universal Adversarial Perturbations for Few-Shot Learning (NeurIPS 2024 · similarity 0.88)
• Data-free Universal Adversarial Perturbation with Pseudo-semantic Prior (CVPR 2025 · similarity 0.88)
• Adversarial Initialization with Universal Adversarial Perturbation: A New Approach to Fast Adversarial Training (AAAI 2024 · similarity 0.86)