Research map / Large language models / Language, safety and interpretability
Watermarking Makes Language Models Radioactive
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that watermarked LLM outputs used as fine-tuning data leave a detectable "radioactive" trace in the trained model, and proposes open- and closed-model statistical tests with de-duplication for reliable p-values.
Problem
Detecting whether a suspect LLM was fine-tuned on synthetic outputs of another LLM, when that output carries a decoding-based watermark, with statistically valid guarantees.
Approach
Treat the watermark as a radioactive signal: score the suspect model's next-token predictions (in open-model reading mode or via prompted closed-model generation) with the watermark detector, and apply de-duplication of scored (k+1)-grams so that the binomial null distribution yields valid p-values even over millions of tokens.
Key results
- In the open-model supervised setting, radioactivity is detected with p < 1e-30 even when only 1% of fine-tuning data is watermarked.
- In the open-model unsupervised setting, the test yields p < 1e-5 when 5% of instruction data is watermarked (log10 p ≈ -5.8 at d=0.1% supervision).
- De-duplication is necessary: without it the closed-model null average p-value is below 1e-30, while with it the null average is 0.42 (close to 0.5).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase supervision from d=0.1% to d=10% | Open-model radioactivity detection · Instruction tuning, ρ=5% | log10(p): -5.8 to <-30 | Helped |
| remove de-duplication (ablation) | Radioactivity detection · Null hypothesis, open and closed model | average p-value: Open: 0.46±0.27 -> 0.053±0.12; Closed: 0.42±0.30 -> <10^{-30} | Hurt |
| increase watermark window from k=1 to k=4 (ablation) | KGW and AK radioactivity detection · Closed model, N=30k, ρ=100% | log10(p): KGW: -43.7±7.4 -> -1.4±0.6; AK: -47.2±4.5 -> -2.8±3.2 | Hurt |
| increase learning rate from 10^-5 to 5·10^-5 or 10^-4 (ablation) | Full fine-tuning · Open model, unsupervised, ρ=100% | log10(p): -32.4 -> -49.6 and -58.0 | Helped |
| reduce fine-tuning from 3 epochs to 1 or 2 epochs (ablation) | Full fine-tuning · Open model, unsupervised, ρ=100% | log10(p): -33.2 -> -20.8 and -29.2 | Hurt |
| replace full fine-tuning with Q-LoRA (ablation) | Radioactivity detection · Open model, unsupervised, ρ=100% | log10(p): -32.4 -> -11.0 | Hurt |
| increase model size from 7B to 13B (ablation) | Full fine-tuning · Open model, unsupervised, ρ=100% | log10(p): -32.4 -> -33.2 | No clear effect |
| add second fine-tuning on OASST1 (ablation) | Radioactivity detection · Open model, unsupervised, ρ=10% | average log10(p): -15 -> -8 | Hurt |
| filter scored k-grams using φ (ablation) | Closed-model radioactivity detection · Supervised, ρ=1%, after scoring 75000 tokens | log10(p): -8 -> -12 | Helped |
| use a different target language rather than English | Closed-model radioactivity detection · Wikipedia prompts, N=250k | log10(p): English: <-50; French: -7.8; Spanish: -5.7; German: -4.0; Catalan: -2.1 | Mixed |
Surprising findings
- Without de-duplication, the null hypothesis test falsely flags a non-watermarked model as radioactive with p < 1e-30 in the closed-model setting.
- Radioactivity is detectable even on watermarked text in a language different from the training data by combining p-values with Fisher's method (<1e-50).
Limitations
- Absolute p-value thresholds (e.g., 5%) depend on watermark scheme and fine-tuning method and are not universal.
- Closed-model detection requires around 1000 or more queries to the suspect model.
- Only radioactivity of decoding-based hash watermarks (KGW, AK) studied; other watermarks not tested.
Experimental setup
Datasets: Self-Instruct generated instructions from Llama-2-chat-7B (watermarked and non-watermarked) OASST1 (second fine-tuning) Wikipedia article beginnings in multiple languages
Benchmarks: Natural Questions TriviaQA GSM8K HumanEval MMLU
Metrics: log10 p-value exact match pass@1 accuracy p-value distribution under null
Baselines: Membership inference attack (perplexity/zlib calibrated KS test) Naive watermark detection on model outputs Synonym-replacement watermark
Builds on: Kirchenbauer et al. green-list watermark (KGW) Aaronson watermark (AK) Self-Instruct Alpaca membership inference attacks
Closest papers by meaning
- PRO: Enabling Precise and Robust Text Watermark for Open-Source LLMs (NeurIPS 2026 · similarity 0.90)
- On the Learnability of Watermarks for Language Models (ICLR 2024 · similarity 0.90)
- Black-Box Detection of Language Model Watermarks (ICLR 2025 · similarity 0.90)
- On the Reliability of Watermarks for Large Language Models (ICLR 2024 · similarity 0.90)
- Can Watermarking Large Language Models Prevent Copyrighted Text Generation and Hide Training Data? (AAAI 2025 · similarity 0.89)
- GaussMark: A Practical Approach for Structural Watermarking of Language Models (ICML 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.