Research map / Adversarial robustness and security / Black-box and transfer attacks
Adversarial Attacks on Data Attribution
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a threat model and two adversarial attacks (Shadow Attack, gray-box; Outlier Attack, black-box) that perturb contributed data to inflate data-attribution-based compensation for image classification and text generation.
Problem
Data attribution methods used to value training data and compensate data providers may be manipulated by adversarial data contributors, but adversarial robustness of these methods has received little systematic study.
Approach
Formalizes a periodic data-contribution threat model with persistence, and proposes Shadow Attack (shadow training plus gradient ascent on Grad-Dot attribution scores) and Outlier Attack (black-box adversarial examples exploiting the fact that outliers are more influential) to inflate top-k compensation share.
Key results
- Shadow Attack inflates compensation share by 206.3% to 456.1% across image classification settings.
- Outlier Attack inflates compensation by 185.2% to 643.9% across four settings, including 262.9% on NanoGPT text generation.
- Random pixel perturbation baseline yields compensation share nearly identical to the original (e.g., 0.0097 vs 0.0098 on setting a).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| perturb contributed data using Shadow Attack | Logistic Regression · MNIST (a) | Compensation Share ratio: 456.1% | Helped |
| perturb contributed data using Shadow Attack | MLP · Digits (b) | Compensation Share ratio: 286.2% | Helped |
| perturb contributed data using Shadow Attack | CNN · MNIST (c) | Compensation Share ratio: 417.0% | Helped |
| perturb contributed data using Shadow Attack | ResNet-18 · CIFAR-10 (d) | Compensation Share ratio: 217.3% | Helped |
| use ResNet-9 instead of ResNet-18 as the shadow model (ablation) | Shadow Attack on ResNet-18 · CIFAR-10 (d) | Compensation Share ratio: 217.3% -> 206.3% | Hurt |
| perturb contributed data using Outlier Attack | Logistic Regression · MNIST (a) | Compensation Share ratio: 643.9% | Helped |
| perturb contributed data using Outlier Attack | MLP · Digits (b) | Compensation Share ratio: 194.6% | Helped |
| perturb contributed data using Outlier Attack | CNN · MNIST (c) | Compensation Share ratio: 596.4% | Helped |
| perturb contributed data using Outlier Attack | ResNet-18 · CIFAR-10 (d) | Compensation Share ratio: 185.2% | Helped |
| perturb contributed text using Outlier Attack | NanoGPT · Shakespeare (e) | Compensation Share ratio: 262.9% | Helped |
| use pixel-wise random perturbation instead of an adversarial attack | Logistic Regression · MNIST (a) | Compensation Share: 0.0098 -> 0.0097 | No clear effect |
| use pixel-wise random perturbation instead of an adversarial attack | CNN · MNIST (c) | Compensation Share: 0.0112 -> 0.0117 | No clear effect |
Surprising findings
- Outlier Attack outperforms Shadow Attack on settings (a) and (c) despite requiring no data distribution knowledge.
- Random perturbation with matched budget yields no compensation gain, indicating the gains come from adversarial design.
Limitations
- Shadow Attack requires knowledge of the training data distribution and training algorithm.
- Outlier Attack is not evaluated under the persistence assumption for generative settings where data distribution is unknown.
- Experiments use small models and datasets (e.g., MLP with 5 hidden layers of 10 neurons, subsets of CIFAR-10).
Experimental setup
Datasets: MNIST Digits CIFAR-10 Shakespeare
Benchmarks: MNIST Digits CIFAR-10 Shakespeare
Metrics: compensation share compensation ratio fraction of change (More/Tied/Fewer)
Baselines: Random Perturbation Original (unperturbed) dataset
Builds on: Influence Function TRAK Grad-Dot Data Shapley TextFooler
Closest papers by meaning
- On the Fragility of Data Attribution When Learning Is Distributed (ICML 2026 · similarity 0.88)
- Rethinking Robustness of Model Attributions (AAAI 2024 · similarity 0.86)
- Taming Hyperparameter Sensitivity in Data Attribution: Practical Selection Without Costly Retraining (NeurIPS 2025 · similarity 0.84)
- RODEO: Robust Outlier Detection via Exposing Adaptive Out-of-Distribution Samples (ICML 2024 · similarity 0.83)
- Adversaries With Incentives: A Strategic Alternative to Adversarial Robustness (ICLR 2025 · similarity 0.83)
- Unlearn and Burn: Adversarial Machine Unlearning Requests Destroy Model Accuracy (ICLR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.