Research map / Large language models / Language, safety and interpretability

StealthInk: A Multi-bit and Stealthy Watermark for Large Language Models

ICML 2025 · Large language models › Language, safety and interpretability · approach: Text watermarking · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes StealthInk, a distribution-preserving (unbiased) multi-bit LLM watermark that embeds provenance bits via red-list reweighting over pseudorandom vocabulary permutations, with position allocation across message chunks.

Problem

Existing LLM watermarks either distort the output distribution or only embed zero-bit information; multi-bit schemes are often non-stealthy and vulnerable to spoofing, and distortion-free ones are fragile to edits.

Approach

A reweighting rule that doubles probabilities of some tokens and zeroes a "red list" interval determined by the message chunk, so the expected watermarked distribution equals the original LLM distribution; a per-query history log of texture keys and position allocation of message chunks (from MPAC) provide capacity and K-shot stealthiness.

Key results

• StealthInk achieves AUC 0.98 and bit accuracy 0.92 when embedding 24 bits in 300 tokens (abstract).
• At 24 bits/200 tokens, StealthInk reaches AUC 0.978 and bit accuracy 0.875 with median PPL 7.141 vs 8.016 non-watermarked, versus MPAC AUC 0.996 with PPL 10.19.
• Under spoofing on Dolly-CW, only 29% of StealthInk-forged texts are detected at TPR@10%FPR, versus much higher rates for MPAC and Qu et al.

Measured effects reported in the paper

Surprising findings

• StealthInk's median perplexity is lower than non-watermarked text, which the authors attribute to filtering low-probability tokens in the evaluation setup.
• Paraphrasing (Dipper, PEGASUS) at epsilon=0.1 drops bit accuracy to around 0.57-0.63 for all schemes, showing multi-bit robustness to paraphrasing remains a challenge.

Limitations

• Paraphrasing attacks substantially reduce bit accuracy for all evaluated multi-bit schemes, including StealthInk.
• Detectability is lower than MPAC at short lengths (e.g., 200 tokens) and requires more tokens to match MPAC with delta=2.
• Stealthiness is only guaranteed in expectation over many generations; per-response PPL and first-token distributions can differ, and repeated same-time queries may leak information.

Experimental setup

Datasets: C4 RealNewsLike Dolly-CW MMW BookReports WMT'14 En-Ro

Benchmarks: C4 RealNewsLike WMT'14 En-Ro Dolly-CW MMW BookReports

Metrics: AUC TPR@1%FPR bit accuracy perplexity BERTScore BLEU ROUGE-1 GPT4 score extraction time

Baselines: MPAC Qu et al. Fernandez et al.

Builds on: Kirchenbauer et al. green-red watermark Dipmark MPAC Christ et al. undetectable watermark Jovanovic et al. watermark stealing

Closest papers by meaning

• PRO: Enabling Precise and Robust Text Watermark for Open-Source LLMs (NeurIPS 2026 · similarity 0.90)
• BiMark: Unbiased Multilayer Watermarking for Large Language Models (ICML 2025 · similarity 0.90)
• Theoretically Grounded Framework for LLM Watermarking: A Distribution-Adaptive Approach (NeurIPS 2025 · similarity 0.90)
• Black-Box Detection of Language Model Watermarks (ICLR 2025 · similarity 0.89)
• Provable Robust Watermarking for AI-Generated Text (ICLR 2024 · similarity 0.89)
• Adaptive Text Watermark for Large Language Models (ICML 2024 · similarity 0.89)