Research map / Large language models / Language, safety and interpretability

STAMP Your Content: Proving Dataset Membership via Watermarked Rephrasings

ICML 2025 · Large language models › Language, safety and interpretability · approach: Text watermarking

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes STAMP, which releases one KGW-watermarked LLM rephrasing of a dataset publicly, keeps several differently keyed rephrasings private, and runs a paired t-test on perplexity differences to prove pretraining membership.

Problem

Content creators and benchmark curators need a reliable third-party way to prove that their dataset was included in LLM pretraining data, without relying on unseen validation sets or machine-unreadable perturbations.

Approach

Use LLM watermarks (KGW) on rephrasings of the same document with distinct hash keys so that public and private versions are generated by the same process; a paired t-test on perplexity differences then tests whether the model prefers the publicly released version, avoiding the bias toward LLM-generated text that confounds human-vs-rephrase comparisons.

Key results

• STAMP reaches p-values of 1.2e-4 (TriviaQA), 2.8e-4 (ARC-C), 7.0e-4 (MMLU) and 6.6e-6 (GSM8K) when each benchmark is injected once and makes up under 0.001% of 6.7B tokens, while PaCoST and LLM DI are significant on at most two benchmarks.
• On off-the-shelf models not trained on the watermarked data (Pythia 1B, Gemma-2 2B/9B, Mistral 7B, LLaMA-3 8B), p-values are all above 0.1 in most cases, showing no false positives in the reported table.
• Watermarked rephrasings keep semantic similarity (P-SP 0.83-0.95) and preserve model rankings on benchmarks, whereas Unicode substitution drops accuracy by up to 20%.

Measured effects reported in the paper

Surprising findings

• Existing rephrasing-based tests that compare human-written text against LLM rephrasings are confounded by LLMs' preference for machine-generated text.
• Membership inference attacks (PPL, Zlib, Min-K%, Min-K%++, DC-PDD) perform near random (AUROC about 0.5) on these benchmarks.

Limitations

• Watermarks must be embedded before content is published, so it cannot be applied to already-released content.
• Requires gray-box access to token probabilities from the model.
• Experiments use continual pretraining of a 1B model rather than training from scratch; rephrasing can introduce errors.

Experimental setup

Datasets: TriviaQA ARC-C MMLU GSM8K OpenWebText EMNLP 2024 abstracts AI Snake Oil newsletter posts

Benchmarks: TriviaQA ARC-C MMLU GSM8K

Metrics: p-value AUROC P-SP semantic similarity accuracy on original vs watermarked benchmarks GPT-4 quality ratings human author study

Baselines: PaCoST LLM DI PPL Zlib Min-K% Min-K%++ DC-PDD Unicode substitution watermark Radioactivity

Builds on: KGW watermark PaCoST LLM DI Waterfall

Closest papers by meaning

• Can Watermarking Large Language Models Prevent Copyrighted Text Generation and Hide Training Data? (AAAI 2025 · similarity 0.89)
• Detecting Pretraining Data from Large Language Models (ICLR 2024 · similarity 0.87)
• Unbiased Watermark for Large Language Models (ICLR 2024 · similarity 0.87)
• Watermarking Makes Language Models Radioactive (NeurIPS 2024 · similarity 0.87)
• SAEMark: Steering Personalized Multilingual LLM Watermarks with Sparse Autoencoders (NeurIPS 2025 · similarity 0.87)
• Break the Trade-off Between Watermark Strength and Speculative Sampling Efficiency for Language Models (ICLR 2026 · similarity 0.87)