Research map / Large language models / Language, safety and interpretability
Can Watermarked LLMs be Identified by Users via Crafted Prompts?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Water-Probe, a black-box method that detects whether an LLM is watermarked by comparing cross-prompt consistency of key-pair output differences, and Water-Bag, a key-bag strategy that makes watermarks harder to identify.
Problem
Whether users can identify if an LLM service is watermarked (imperceptibility of watermarked LLMs themselves) when providers do not disclose watermarks, across both n-gram based and fixed-key-list based generation-time watermarks.
Approach
Craft pairs of highly correlated prompts that force repeated sampling under the same simulated watermark keys (via random prefixes); watermarks induce consistent key-pair output differences across the prompts, measured by rank-transformed cosine similarity and a z-test. Water-Bag merges multiple master keys and their inversions so that the average logit effect is distortion-free and per-generation
Key results
- Water-Probe-v2 with 10^4 samples identifies all tested watermarks (e.g., Aar average similarity 0.813 vs 0.026 for non-watermarked LLMs), while Water-Probe-v1 is ineffective on fixed-key-list watermarks.
- Water-Probe-v2 achieves average similarity 0.729 on ITS-Edit and 0.424 on EXP-Edit, with near-zero (≤0.03) similarity for non-watermarked models.
- Water-Bag with bag size |K∪K̄|=2 or 4 on KGW lowers Water-Probe-v2 similarity to 0.25-0.34 while keeping detection F1 at 1.0 on C4, at the cost of robustness (GPT-3.5 rewrite F1 0.748 vs 0.843).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Water-Probe-v1 for identifying n-gram watermarks rather than non-watermarked models | Water-Probe-v1 · Average across tested LLMs; n-gram watermark conditions vs non-watermarked | similarity: 0.029 -> 0.854 | Helped |
| Use Water-Probe-v1 on fixed-key-list watermarks | Water-Probe-v1 · EXP-Edit and ITS-Edit | similarity: 0.029 -> 0.031 (EXP-Edit); 0.029 -> 0.032 (ITS-Edit) | No clear effect |
| Use Water-Probe-v2 to identify fixed-key-list watermarks | Water-Probe-v2 · EXP-Edit and ITS-Edit | similarity: 0.026 -> 0.424 (EXP-Edit); 0.026 -> 0.729 (ITS-Edit) | Helped |
| Increase Water-Bag key-bag size from 1 to 8 (ablation) | KGW Water-Bag with Water-Probe-v2 (n=3) · Watermarked LLM identification | similarity: 0.34 -> 0.16 | Hurt |
| Increase Water-Bag key-bag size from 1 to 8 (ablation) | KGW Water-Bag · Watermarked text detection | Detection F1-score: 1.0 -> 1.0 | No clear effect |
| Increase sampling prefix length from n=3 to n=5 (ablation) | Water-Probe-v2 on KGW Water-Bag · Watermarked LLM identification | similarity: 0.34 -> 0.32 (|K∪K̄|=1); 0.16 -> 0.07 (|K∪K̄|=8) | Mixed |
Surprising findings
- Water-Bag with bag size 2 makes Water-Probe-v1 fail (similarity 0.05) on KGW while KGW alone is detected at 0.42.
- Even distortion-free watermarks (Aar, DiPMark, gamma-reweight) are identifiable by Water-Probe.
Limitations
- Water-Probe-v2 requires many samples (10^4) and Water-Probe-v1 is restricted to n-gram watermarks.
- Increasing key randomness with Water-Bag reduces watermarked text robustness to rewriting.
- Global fixed-key (Unigram) identification relies on a prior distribution and is not theoretically guaranteed.
Experimental setup
Datasets: C4
Benchmarks: C4
Metrics: z-score average similarity F1-score perplexity detection time
Baselines: KGW Aar KGW-Min KGW-Skip DiPMark gamma-reweight EXP-Edit ITS-Edit Water-Probe-v1
Builds on: KGW Aar EXP-Edit DiPMark Unigram
Closest papers by meaning
- Black-Box Detection of Language Model Watermarks (ICLR 2025 · similarity 0.90)
- PRO: Enabling Precise and Robust Text Watermark for Open-Source LLMs (NeurIPS 2026 · similarity 0.90)
- No Free Lunch in LLM Watermarking: Trade-offs in Watermarking Design Choices (NeurIPS 2024 · similarity 0.90)
- Discovering Spoofing Attempts on Language Model Watermarks (ICML 2025 · similarity 0.89)
- Can Watermarking Large Language Models Prevent Copyrighted Text Generation and Hide Training Data? (AAAI 2025 · similarity 0.89)
- Watermark Stealing in Large Language Models (ICML 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.