Research map / Adversarial robustness and security / Certified and empirical robustness
Contrasting Adversarial Perturbations: The Space of Harmless Perturbations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that DNNs have harmless perturbation subspaces (the nullspace of linear layers and expansions from non-injective nonlinearities) under which network outputs are exactly unchanged, and uses them for privacy-preserving images and model fingerprints.
Problem
Whether DNNs admit input perturbations of arbitrary magnitude that leave the network output strictly invariant, in contrast to the well-studied vulnerability to small adversarial perturbations.
Approach
Harmless perturbations are the nullspace of a linear layer's parameter matrix when input dimension exceeds output dimension; any perturbation decomposes into harmless and orthogonal (harmful) components, so its output equals that of the orthogonal component regardless of magnitude. Non-injective nonlinearities (ReLU, max/average pooling, softmax) may expand this space; the approach is applied to p
Key results
- Harmless perturbations on CIFAR-10 yield output RMSE around 3.6e-15 to 8.6e-15 across magnitudes up to 32 epsilon, versus 6.2 to 11.3 for adversarial perturbations and 0.12 to 5.0 for Gaussian noise.
- For privacy protection with ResNet-50 on CIFAR-10, harmless perturbations give SSIM 0.4719 (vs 0.6825 for Gaussian noise) and 0.00% accuracy change (vs 32.46% for Gaussian noise).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace Gaussian noise with a harmless perturbation at ε | ResNet-50 · CIFAR-10 | RMSE: 0.1226 -> 3.63e-15 | Helped |
| Use a harmless perturbation instead of Gaussian noise | ResNet-50 · CIFAR-10 | Δaccuracy: 32.46% -> 0.00% | Helped |
| Use a harmless perturbation instead of Gaussian noise | ResNet-50 · CIFAR-10 | SSIM: 0.6825 -> 0.4719 | Helped |
| Increase harmless perturbation magnitude from ε to 32ε | ResNet-50 · CIFAR-10 | RMSE: 3.63e-15 -> 8.55e-15 | Mixed |
| Increase least harmful perturbation magnitude from ε to 32ε | ResNet-50 · CIFAR-10 | RMSE: 0.0003 -> 0.0105 | Hurt |
Surprising findings
- Harmless perturbations of arbitrary large magnitude leave DNN outputs unchanged while being visually unrecognizable to humans.
- Distance-based similarity metrics such as Euclidean and cosine distance can be invalidated, since inputs differing by harmless perturbations are deemed dissimilar yet yield identical network outputs.
Limitations
- For linear layers with input dimension not exceeding output dimension, no non-zero harmless perturbation exists; only least harmful perturbations can be computed.
- Harmless perturbation space for non-injective nonlinearities depends on input samples and may expand only under conditions, so the expansion is not guaranteed.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 MNIST SHVN
Benchmarks: CIFAR-10 CIFAR-100 MNIST SHVN
Metrics: root mean squared error SSIM LPIPS accuracy change
Baselines: Gaussian noise adversarial perturbation
Builds on: adversarial examples adversarial training
Closest papers by meaning
- Learning perturbation sets for robust machine learning (ICLR 2021 · similarity 0.91)
- Layer-Aware Analysis of Catastrophic Overfitting: Revealing the Pseudo-Robust Shortcut Dependency (ICML 2024 · similarity 0.88)
- Wide Two-Layer Networks can Learn from Adversarial Perturbations (NeurIPS 2024 · similarity 0.87)
- Theoretical Understanding of Learning from Adversarial Perturbations (ICLR 2024 · similarity 0.87)
- Verification of Neural Networks Against Convolutional Perturbations via Parameterised Kernels (AAAI 2025 · similarity 0.87)
- Towards Transferable Adversarial Attacks with Centralized Perturbation (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.