Research map / Large language models / Language, safety and interpretability
Adaptive Text Watermark for Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes an adaptive LLM text watermark that perturbs only high-entropy tokens (entropy estimated by a small auxiliary model) and scales logits by a semantic-embedding-driven vector, improving quality, robustness and security over KGW and EXP-edit.
Problem
Watermarking LLM text while jointly preserving robustness to paraphrasing, security against spoofing/decryption, and text quality, and detecting without access to the original prompt or model.
Approach
Adaptively watermark only high-entropy tokens (entropy from a small measurement model), derive the green/red logits scaling vector from the semantic embedding of previous text via a trained semantic mapping network, and inject the watermark by scaling logits (temperature-like) instead of adding a fixed bias.
Key results
- Under GPT-3.5 paraphrase on OPT-6.7, the method reaches ROC-AUC 0.981 and Best F1 0.932, above KGW-0 (0.964/0.904) and EXP-edit (0.881/0.801).
- Spoofing decryption rate on OPT-6.7 under a strengthened attack is 0.571, versus 0.963 for KGW-0 and 0.912 for KGW-1.
- Perplexity stays close to un-watermarked text; with alpha=0 (no AWTI) median perplexity exceeds 10, while alpha=2 reduces it substantially.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Adaptive Watermark instead of KGW-0 | watermarking on OPT-6.7B · C4 realnewslike, GPT-3.5 paraphrase | ROC-AUC: 0.964 -> 0.981 | Helped |
| Use Adaptive Watermark instead of KGW-0 | watermarking on OPT-6.7B · C4 realnewslike, DIPPER paraphrase | ROC-AUC: 0.936 -> 0.964 | Helped |
| Use Adaptive Watermark instead of KGW-0 | watermarking on OPT-6.7B · ELI5, GPT-3 paraphrase | ROC-AUC: 0.971 -> 0.967 | Hurt |
| Apply AWTI with entropy threshold α=1 rather than skip AWTI at α=0 (ablation) | KGW-0 watermark on OPT-6.7B · C4 realnewslike, GPT-3.5 paraphrase | ROC-AUC: 0.978 -> 0.988 | Helped |
| Apply AWTI at α=1 instead of α=0 (ablation) | KGW-0 watermark on OPT-6.7B · C4 realnewslike | perplexity | Helped |
| Increase watermark strength from δ=0.5 to δ=2 (ablation) | Adaptive Watermark on OPT-6.7B · C4 realnewslike, GPT-3.5 paraphrase | ROC-AUC: 0.982 -> 0.978 | Hurt |
| Use GPT2-large or OPT-350M rather than GPT2 or OPT-125M as measurement model (ablation) | Adaptive Watermark on OPT-6.7B · C4 realnewslike | ROC-AUC: 0.985; 0.981; 0.987; 0.982 | Mixed |
| Add AWTI to KGW-0 (ablation) | KGW-0 watermark on OPT-6.7B · C4 realnewslike, GPT-3 paraphrase | ROC-AUC: 0.964 -> 0.962 | No clear effect |
| Add SLSVE to KGW-0 + AWTI (ablation) | KGW-0 watermark on OPT-6.7B · C4 realnewslike, GPT-3 paraphrase | ROC-AUC: 0.962 -> 0.960 | Hurt |
| Add AWTS to KGW-0 + AWTI + SLSVE (ablation) | KGW-0 watermark on OPT-6.7B · C4 realnewslike, GPT-3 paraphrase | ROC-AUC: 0.960 -> 0.981 | Helped |
Surprising findings
- Perplexity of the watermarked text decreases as watermark strength delta increases within a reasonable range, attributed to temperature-based scaling.
- Under the strengthened spoofing attack with a fixed semantic embedding, the decryption rate is only an upper bound for practical attacks.
Limitations
- Robustness gains on ROC-AUC are modest relative to some baselines in the no-attack setting, and performance depends on per-model tuning of entropy threshold and watermark strength.
- Security evaluation uses a strengthened spoofing attack with fixed semantics, so it is an upper bound rather than a full security proof.
- Detection is slower than KGW variants due to the auxiliary models.
Experimental setup
Datasets: C4 realnewslike ELI5 Sentence-Compression
Benchmarks: C4 realnewslike ELI5
Metrics: ROC-AUC Best F1 Score perplexity TPR@1%FPR TPR@10%FPR decryption rate repetition rate average watermark rate
Baselines: KGW-0 KGW-1 KGW-2 EXP-edit SWEET SIR
Builds on: KGW watermark EXP-edit Sentence-BERT SWEET SIR
Closest papers by meaning
- Theoretically Grounded Framework for LLM Watermarking: A Distribution-Adaptive Approach (NeurIPS 2025 · similarity 0.91)
- A Semantic Invariant Robust Watermark for Large Language Models (ICLR 2024 · similarity 0.91)
- Provable Robust Watermarking for AI-Generated Text (ICLR 2024 · similarity 0.91)
- Optimizing Adaptive Attacks against Watermarks for Language Models (ICML 2025 · similarity 0.90)
- PRO: Enabling Precise and Robust Text Watermark for Open-Source LLMs (NeurIPS 2026 · similarity 0.90)
- Learning to Watermark: A Selective Watermarking Framework for Large Language Models via Multi-Objective Optimization (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.