Research map / Large language models / Language, safety and interpretability

BiMark: Unbiased Multilayer Watermarking for Large Language Models

ICML 2025 · Large language models › Language, safety and interpretability · approach: Text watermarking · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes BiMark, an unbiased multi-bit LLM watermark that combines token-counting bit-flip reweighting, multilayer reweighting, and XOR-based one-time-pad message encoding to embed messages with model-agnostic, message-agnostic detection.

Problem

Existing LLM watermarks struggle to jointly preserve text quality, allow model-agnostic detection, and embed multi-bit messages; unbiased methods rely on cumulative-probability partitions needing model access, while multi-bit methods are built on biased Soft Red List.

Approach

A fair coin flip decides which of two equal vocabulary bipartitions is boosted (bit-flip unbiased reweighting), stacked over multiple independent layers; message bits are XORed with pseudorandom one-time-pad masks to produce the coin flips, so message bits can be recovered by token-membership voting without the model or message space.

Key results

• For 50-token texts, BiMark improves message extraction rate over MPAC by 20.87%, 29.50%, and 30.02% for 8-, 16-, and 32-bit messages.
• On CNN/DailyMail summarization, BiMark's BERTScore 32.48 and ROUGE-1 38.32 match the no-watermark baseline (32.45 / 38.32), while Soft Red List (2.0) drops to 31.15 / 36.99.
• On WMT'16 En-Ro, BiMark achieves BLEU 22.15 versus 21.93 without watermark, the best among watermarking methods tested.

Measured effects reported in the paper

Surprising findings

• Multilayer reweighting shows shallow layers dominate the final distribution, and detectability peaks at an intermediate layer count (d=10) before declining with more layers.

Limitations

• Multilayer design adds per-token computation proportional to the number of layers d.
• Detectability for very short texts and under strong paraphrasing remains limited; the paper reports lower extraction against DIPPER paraphrasing than SynthID on some settings.
• Type-II error analysis shows detection strength depends on output entropy, with effectiveness peaking at intermediate tau.

Experimental setup

Datasets: C4-RealNewslike CNN/DailyMail WMT'16 En-Ro

Benchmarks: CNN/DailyMail WMT'16 En-Ro

Metrics: message extraction rate perplexity TPR at 1% FPR BERTScore-F1 ROUGE-1 BLEU z-score

Baselines: MPAC Soft Red List SynthID Gumbel Sampling γ-Reweight DiPmark

Builds on: Soft Red List MPAC unbiased reweighting (Kuditipudi-style undetectable watermark) DiPmark SynthID

Closest papers by meaning

• GaussMark: A Practical Approach for Structural Watermarking of Language Models (ICML 2025 · similarity 0.91)
• ArcMark: Distortion-Free Multi-Byte LLM Watermark via Optimal Transport (NeurIPS 2026 · similarity 0.91)
• PRO: Enabling Precise and Robust Text Watermark for Open-Source LLMs (NeurIPS 2026 · similarity 0.91)
• Theoretically Grounded Framework for LLM Watermarking: A Distribution-Adaptive Approach (NeurIPS 2025 · similarity 0.91)
• Black-Box Detection of Language Model Watermarks (ICLR 2025 · similarity 0.90)
• Majority Bit-Aware Watermarking for Large Language Models (NeurIPS 2026 · similarity 0.90)