Research map / Large language models / Language, safety and interpretability
Segmenting Watermarked Texts From Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes randomization-test p-values over sliding windows plus SeedBS-NOT change point detection to detect and segment watermarked sub-strings in LLM text edited by insertions/substitutions.
Problem
Detecting whether a published (possibly edited) text is LLM-generated with a watermark, and separating it into watermarked and non-watermarked sub-strings.
Approach
Convert a text into a sequence of sliding-window randomization-test p-values, then treat watermarked/non-watermarked segmentation as a change point detection problem solved by seeded binary segmentation with narrowest-over-threshold selection and a moving block bootstrap for significance.
Key results
- EMS achieves the highest Rand index across Settings 2-4, outperforming ITS, ITSL and EMSL in the GPT-2 experiments.
- Theory: randomization test gives exact Type I error P(p_T<=alpha)=floor((T+1)alpha)/(T+1) under the null, and power tends to 1 under conditions.
- Change point location estimator satisfies |tau_hat - tau*| = O_p(sqrt(mB log(m/B)) / D), where D is the KS distance.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use EMS instead of ITS or ITSL | Watermark segmentation on GPT2, Settings 2–4 · C4 insertion and substitution attacks (Settings 2–4) | Rand index | Helped |
| Use EMS instead of other evaluated methods | Watermark segmentation on GPT2 · C4 insertion and substitution attacks (Settings 2–4) | Rand index | Helped |
| Use EMS or EMSL instead of ITS or ITSL | Change point detection on GPT2 · C4, no-change-point setting (Setting 1) | false discoveries | Helped |
| Use EMS instead of other evaluated methods | Watermark segmentation on OPT-1.3b · C4, Settings 2–4 | Rand index | Helped |
| Lower the detection threshold | Change point detection on OPT-1.3b · C4, no-change-point setting (Setting 1) | false discoveries | Helped |
Surprising findings
- Inverse transform sampling methods produce more false change point detections than exponential minimum sampling methods on unwatermarked-attack-free text (Setting 1).
Limitations
- Performance depends heavily on quality of the randomization-based p-value sequence.
- Theory assumes watermark key independence and specific conditions; analysis for single change point.
- Only insertion and substitution attacks evaluated; paraphrase-type edits not studied.
Experimental setup
Datasets: Google C4 (news-like subset)
Benchmarks: Google C4 news-like prompts with simulated insertion/substitution attacks
Metrics: Type I error / false detections Rand index change point location error
Baselines: ITS ITSL EMS EMSL
Builds on: inverse transform sampling watermark exponential minimum sampling watermark wild binary segmentation seeded binary segmentation Levenshtein distance
Closest papers by meaning
- Black-Box Detection of Language Model Watermarks (ICLR 2025 · similarity 0.88)
- GaussMark: A Practical Approach for Structural Watermarking of Language Models (ICML 2025 · similarity 0.88)
- PRO: Enabling Precise and Robust Text Watermark for Open-Source LLMs (NeurIPS 2026 · similarity 0.88)
- On the Learnability of Watermarks for Language Models (ICLR 2024 · similarity 0.88)
- Discovering Spoofing Attempts on Language Model Watermarks (ICML 2025 · similarity 0.88)
- Can Watermarking Large Language Models Prevent Copyrighted Text Generation and Hide Training Data? (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.