Research map / Large language models / Language, safety and interpretability
On the Learnability of Watermarks for Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes watermark distillation to train LMs (Llama 2 7B, Pythia 1.4B) to natively generate KGW, Aaronson, and KTH watermarked text under standard decoding, and shows a proof-of-concept spoofing attack.
Problem
Whether language models can directly learn to generate watermarked text under their natural sampling distribution (weights-based watermarking), without a specialized decoder, and what this implies for open-model watermarking and watermark spoofing.
Approach
Logit-based and sampling-based watermark distillation train a student LM to match or imitate a teacher that uses decoding-based watermarking, so the watermark is embedded in the weights and appears under standard decoding.
Key results
- Higher-distortion watermarks are learned with median p-values below 0.0001 (e.g., KGW k=0 delta=2 and Aar k=2 match decoding-based p-values) under both distillation methods.
- Sampling-based distillation of Llama 2-Chat 7B's KGW watermark into Alpaca-7B yields a 71% harmful response rate for the adversary vs 0% for the victim, with median p-value 0.002 on harmful responses.
- Weights-based watermarks are removed by fine-tuning on OpenWebText, and lower-distortion watermarks need more samples (KGW k=2 delta=2 median p-value drops from 0.1 to 0.012 with 5x training).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use logit-based or sampling-based watermark distillation instead of only decoding-time watermarking | Llama 2 7B · C4 RealNewsLike, higher-distortion KGW, Aar, and KTH settings | watermark detectability: median p-values less than 0.0001 | Helped |
| Use lower-distortion watermark settings (ablation) | Logit and sampling watermark distillation · C4 RealNewsLike | median detection p-value: 9e-18 -> 1e-01 | Hurt |
| Train KGW k=2, δ=2 logit distillation for five times longer on more data (ablation) | Llama 2 7B logit distillation · KGW k=2, δ=2 | median detection p-value: 0.1 -> 0.012 | Helped |
| Increase training samples and steps (ablation) | Logit and sampling watermark distillation · C4 RealNewsLike | median detection p-value | Helped |
| Use sampling-based rather than logit-based distillation | Aar watermark distillation · C4 RealNewsLike, Aar | seq-rep-3 | Mixed |
| Use sampling-based rather than logit-based distillation | KTH watermark distillation · C4 RealNewsLike, especially larger shift counts | watermark detectability | Helped |
| Apply further fine-tuning on normal, non-watermarked text (ablation) | Logit-distilled Llama 2 7B · OpenWebText | watermark detectability | Hurt |
| Change decoding parameters, including nucleus threshold or temperature (ablation) | Watermark-distilled Llama 2 7B · KGW, Aar, and KTH generations | median detection p-value: 7e-09 -> 1e-12 | No clear effect |
| Increase random token edits (ablation) | Watermark-distilled Llama 2 7B · Edited generated text | watermark detectability | Hurt |
| Use KTH rather than KGW or Aar under high edit proportions | Watermark-distilled models · Edited generated text, around 60--70% edits | watermark detectability | Helped |
| Use two watermark keys for half of the sampling-distillation training samples each (ablation) | Sampling-distilled Llama 2 7B · KGW k=1, Aar k=2, and KTH s=1 | median detection p-value: 8e-07 -> 1e-02; 3e-17 -> 3e-03; 1e-04 -> 8e-02 | Hurt |
| Distill a victim watermark into a model that responds to harmful prompts | Alpaca-7B adversary using Llama 2-Chat 7B samples · HarmfulQ | harmful response rate: 80% -> 71% | Mixed |
Surprising findings
- Watermarks learned by distillation are lost after fine-tuning on ordinary non-watermarked text.
- Using two different keys during sampling-based distillation still yields learned watermarks for both keys, with larger p-values.
Limitations
- Weights-based watermarks are not robust to fine-tuning on normal text.
- Lower-distortion watermarks are less sample efficient and learned less strongly.
- Distilled models have a worse generation quality and detectability tradeoff than decoding-based watermarking.
Experimental setup
Datasets: OpenWebText C4 RealNewsLike Wikipedia articles arXiv papers HarmfulQ
Benchmarks: C4 RealNewsLike Wikipedia articles arXiv papers HarmfulQ
Metrics: median watermark detection p-value median KTH test statistic AUROC perplexity (Llama 2 13B) seq-rep-3 harmful response rate
Baselines: KGW decoding-based watermark Aaronson decoding-based watermark KTH decoding-based watermark Base student
Builds on: KGW watermark Aaronson watermark KTH watermark knowledge distillation
Closest papers by meaning
- PRO: Enabling Precise and Robust Text Watermark for Open-Source LLMs (NeurIPS 2026 · similarity 0.91)
- Theoretically Grounded Framework for LLM Watermarking: A Distribution-Adaptive Approach (NeurIPS 2025 · similarity 0.91)
- GaussMark: A Practical Approach for Structural Watermarking of Language Models (ICML 2025 · similarity 0.90)
- Watermarks in the Sand: Impossibility of Strong Watermarking for Language Models (ICML 2024 · similarity 0.90)
- Break the Trade-off Between Watermark Strength and Speculative Sampling Efficiency for Language Models (ICLR 2026 · similarity 0.90)
- Discovering Spoofing Attempts on Language Model Watermarks (ICML 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.