Research map / Adversarial robustness and security / Attacks on text and graphs
Provable Watermarking for Data Poisoning Attacks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes post-poisoning and poisoning-concurrent watermarking for data poisoning attacks, with theoretical length conditions for watermark detectability and poisoning utility, validated on Narcissus, AdvSc, UE and AP attacks.
Problem
Harmless or purposeful data poisoning (e.g., dataset ownership verification, unlearnable examples for artists) can be misused by innocent users, so poisoners need a provable, verifiable way to disclose that a dataset has been poisoned.
Approach
Embed a key-detectable watermark (a low-dimensional additive perturbation whose inner product with a random ±1 key separates poisoned from clean data) either after poisoning by a third party or jointly with poisoning on disjoint dimensions, and prove detectability and poisoning utility under length thresholds Θ(√d/ε_w) and Θ(1/ε_w²) to O(√d/ε_p).
Key results
- On CIFAR-10 with ResNet-18, Narcissus under poisoning-concurrent watermarking reaches AUROC 0.9968 at watermark length 500 and 1.0000 at 1500, while ASR falls to 22.46% at length 2000 and 11.86% at 2500.
- For UE under poisoning-concurrent watermarking, test accuracy recovers to 88.78% at length 2500 (from ~10% baseline), indicating poisoning becomes ineffective at large watermark lengths.
- Post-poisoning watermarking keeps attack strength for Narcissus (ASR ~90-93%) across lengths up to 3000, while AUROC rises from 0.5522 at length 100 to 1.0000 at length 2000.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase watermark length from q=100 to q=3000 (ablation) | Narcissus post-poisoning watermark · CIFAR-10 (ResNet-18) | AUROC: 0.5522 -> 1.0000 | Helped |
| Increase watermark length from q=100 to q=3000 (ablation) | Narcissus poisoning-concurrent watermark · CIFAR-10 (ResNet-18) | AUROC: 0.9294 -> 1.0000 | Helped |
| Increase watermark length from q=100 to q=3000 (ablation) | AdvSc post-poisoning watermark · CIFAR-10 (ResNet-18) | AUROC: 0.8036 -> 1.0000 | Helped |
| Increase watermark length from q=100 to q=3000 (ablation) | AdvSc poisoning-concurrent watermark · CIFAR-10 (ResNet-18) | AUROC: 0.9679 -> 1.0000 | Helped |
| Increase watermark length from q=100 to q=3000 (ablation) | UE post-poisoning watermark · CIFAR-10 (ResNet-18) | AUROC: 0.5844 -> 1.0000 | Helped |
| Increase watermark length from q=100 to q=3000 (ablation) | UE poisoning-concurrent watermark · CIFAR-10 (ResNet-18) | AUROC: 0.8197 -> 1.0000 | Helped |
| Increase watermark length from q=100 to q=3000 (ablation) | AP post-poisoning watermark · CIFAR-10 (ResNet-18) | AUROC: 0.5688 -> 1.0000 | Helped |
| Increase watermark length from q=100 to q=3000 (ablation) | AP poisoning-concurrent watermark · CIFAR-10 (ResNet-18) | AUROC: 0.6950 -> 1.0000 | Helped |
| Increase q to 3000 (ablation) | Narcissus poisoning-concurrent watermark · CIFAR-10 (ResNet-18) | ASR (%): 95.04 -> 9.75 | Hurt |
| Increase q to 3000 (ablation) | AdvSc poisoning-concurrent watermark · CIFAR-10 (ResNet-18) | ASR (%): 95.53 -> 9.97 | Hurt |
| Increase q to 3000 (ablation) | Narcissus post-poisoning watermark · CIFAR-10 (ResNet-18) | ASR (%): 95.04 -> 90.02 | No clear effect |
| Increase q to 3000 (ablation) | AdvSc post-poisoning watermark · CIFAR-10 (ResNet-18) | ASR (%): 95.53 -> 74.82 | Hurt |
Surprising findings
- Poisoning-concurrent watermarking can cause the poisoning to fail entirely at large watermark lengths (e.g., UE accuracy back to ~90%), so the watermark length must stay below O(sqrt(d)/eps_p).
- AdvSc ASR under post-poisoning watermarking drops at large lengths, attributed to its reliance on shortcuts in the top-left quarter of the image.
Limitations
- Necessary conditions for detectability and poisoning utility are left open.
- Watermark design is simple (random dimensions, sign key); robustness against removal is only discussed in appendix and not optimized.
- Theory assumes Xavier-normalized feed-forward networks and i.i.d. data, while experiments use ResNet-style architectures.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 Tiny-ImageNet
Benchmarks: CIFAR-10 CIFAR-100 Tiny-ImageNet
Metrics: clean accuracy attack success rate AUROC
Baselines: Narcissus AdvSc UE AP
Builds on: Narcissus AdvSc unlearnable examples McDiarmid's inequality Neural Tangent Kernel
Closest papers by meaning
- Exploring the Limits of Model-Targeted Indiscriminate Data Poisoning Attacks (ICML 2023 · similarity 0.87)
- Activation Gradient based Poisoned Sample Detection Against Backdoor Attacks (ICLR 2025 · similarity 0.86)
- DataElixir: Purifying Poisoned Dataset to Mitigate Backdoor Attacks via Diffusion Models (AAAI 2024 · similarity 0.86)
- Phantom Transfer: Data Poisoning can Survive Data-Level Defences (NeurIPS 2026 · similarity 0.85)
- Data Taggants: Dataset Ownership Verification Via Harmless Targeted Data Poisoning (ICLR 2025 · similarity 0.85)
- From Trojan Horses to Castle Walls: Unveiling Bilateral Data Poisoning Effects in Diffusion Models (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.