Research map / Large language models / Language, safety and interpretability
Discovering Spoofing Attempts on Language Model Watermarks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes statistical tests (Standard and Reprompting, using Pearson/Spearman correlation between watermark color and n-gram/unigram scores) that detect spoofed text forged by learning-based spoofers (Stealing, Distillation) against Red-Green watermarks.
Problem
Learning-based spoofing attacks can forge LLM watermarks and pass the detector, and prior work had not studied post-hoc detection of such spoofing attempts.
Approach
Spoofed text contains artifacts because a spoofer only knows the green/red split for n-grams seen in its training data; the paper correlates observed token colors with context-frequency scores (or unigram frequency scores) to build hypothesis tests whose null distribution is controlled, using reprompting to correct for the LM-induced dependence.
Key results
- For Stealing (h=2) at T=3000 with the Reprompting test, TPR@1% reaches about 0.97 at 5% FPR-level results; the paper reports above 90% TPR at 1% FPR for all tested scenarios at T=3000.
- Type 1 error is controlled: rejection rates on unspoofed Llama2-7B watermarked text align with the identity line across h and T.
- Tests generalize to SelfHash and to AAR and KTH schemes (via Distillation), and the test power is similar across spoofer LMs, including a spoofer using the same model as the provider.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase the input length from T=500 to T=3000 | Reprompting detection with (h+1)-gram score on KGW SumHash Stealing, h=1 · KGW SumHash, Stealing, h=1 | TPR @1%: 0.62 -> 1.00 | Helped |
| Increase the input length from T=500 to T=3000 | Reprompting detection with (h+1)-gram score on KGW SumHash Distillation, h=1 · KGW SumHash, Distillation, h=1 | TPR @1%: 0.48 -> 1.00 | Helped |
| Use Reprompting with (h+1)-gram score instead of Standard with unigram score (ablation) | Spoofing detection on KGW SumHash Stealing, h=3 · KGW SumHash, Stealing, h=3 | TPR @1%: 0.27 -> 0.47 | Helped |
| Use the Standard method with unigram score instead of Reprompting with (h+1)-gram score (ablation) | Spoofing detection on KGW SumHash Stealing, h=3 · KGW SumHash, Stealing, h=3 | TPR @1%: 0.47 -> 0.27 | Hurt |
| Use Standard with unigram score instead of Reprompting with (h+1)-gram score (ablation) | Spoofing detection on SelfHash Stealing, h=3 · SelfHash, Stealing, h=3 | Rejection rate @1%, T=2000: 0.99 -> 0.32 | Hurt |
| Estimate the reprompting input from the received text instead of using the original prompt (ablation) | Reprompting detection on KGW SumHash Stealing, h=1 · KGW SumHash, Stealing, h=1 | TPR @1% FPR: 5% | Hurt |
| Increase the spoofer training dataset size by 9 times (ablation) | Distillation spoofing detection, LeftHash, h=1 · LeftHash, Distillation, T=1000 | TPR @0.1%: 94 percent -> 87 percent | Hurt |
| Replace per-text Reprompting with a fixed corpus estimate of the watermarked correlation (ablation) | Spoofing detection on KGW SumHash Distillation, h=2 · KGW SumHash, Distillation, h=2 | False positive rate | Hurt |
Surprising findings
- A spoofer using the same LM as the model owner does not significantly lower test power, suggesting artifacts reflect the spoofer's limited knowledge rather than model mismatch.
- Increasing the size of the spoofer training data only mildly reduces test power (94% to 87% TPR at 0.1% FPR for a 9x increase), so it is not an effective way to hide artifacts.
Limitations
- No theoretical guarantee of power; tests rely on assumptions about learning-based spoofers.
- Adaptive spoofers that learn the vocabulary split may avoid leaving the artifacts.
- Requires sufficiently long total text length for high power.
Experimental setup
Datasets: C4 Dolly Wikipedia Repliqa Math OpenWebText
Benchmarks: C4 prompts continuations (spoofed vs watermarked) Dolly prompts
Metrics: false positive rate true positive rate rejection rate p-value calibration
Baselines: Stealing Distillation
Builds on: KGW watermark Stealing spoofing attack Distillation spoofing attack AAR watermark C4
Closest papers by meaning
- Watermark Stealing in Large Language Models (ICML 2024 · similarity 0.93)
- Enhancing LLM Watermark Resilience Against Both Scrubbing and Spoofing Attacks (NeurIPS 2025 · similarity 0.90)
- On the Learnability of Watermarks for Language Models (ICLR 2024 · similarity 0.90)
- Black-Box Detection of Language Model Watermarks (ICLR 2025 · similarity 0.90)
- Can Watermarking Large Language Models Prevent Copyrighted Text Generation and Hide Training Data? (AAAI 2025 · similarity 0.90)
- Can Watermarked LLMs be Identified by Users via Crafted Prompts? (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.