Research map / Large language models / Language, safety and interpretability
SAEMark: Steering Personalized Multilingual LLM Watermarks with Sparse Autoencoders
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SAEMark, an inference-time, black-box multi-bit LLM watermark that selects among sampled text units whose sparse-autoencoder-based Feature Concentration Score matches key-derived targets.
Problem
Watermarking LLM text for personalized multi-bit attribution while preserving quality, working with API-only closed models, and generalizing across languages and domains (English, Chinese, code).
Approach
Instead of modifying logits or rewriting text, sample N candidate units from the LLM and select the one whose normalized SAE-based feature statistic (FCS) is closest to a key-derived target value; the SAE runs on a separate anchor model so the target LLM is only queried via text output.
Key results
- SAEMark reaches 99.7% F1 on English (C4), 99.2% on Chinese (LCSTS), and 66.3% on code (MBPP), versus 93.2%, 95.1%, and 11.6% for multi-bit Waterfall on the same datasets.
- On PandaLM, SAEMark has the highest quality win-rate (67.6%) among watermarks as judged by GPT-4o.
- Reports over 90% accuracy up to 10 bits (1,024 users) and 75% at 13 bits at fixed text length.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SAEMark instead of SWEET | Watermarking on MBPP · MBPP | F1: 66.3% vs. 62.4% | Helped |
| Use SAEMark instead of Waterfall | Multi-bit watermarking · C4 | F1: 99.7% vs. 93.2% | Helped |
| Use SAEMark instead of Waterfall | Multi-bit watermarking · MBPP | F1: 66.3% vs. 11.6% | Helped |
| Use SAEMark instead of Waterfall | Watermarking on PandaLM · PandaLM | Quality: 67.6% | Helped |
| Select among naturally generated candidates with SAEMark rather than use unwatermarked text | Qwen2.5-7B-Instruct on BIGGen-Bench · BIGGen-Bench | Pointwise quality score: 4.13 -> 4.05 | Hurt |
| Select among naturally generated candidates with SAEMark rather than use unwatermarked text | Llama-3.2-3B-Instruct on BIGGen-Bench · BIGGen-Bench | Pointwise quality score: 3.69 -> 3.85 | Helped |
| Select among naturally generated candidates with SAEMark rather than use unwatermarked text | gemma-3-4b-it on BIGGen-Bench · BIGGen-Bench | Pointwise quality score: 4.26 -> 4.23 | Hurt |
| Reduce candidate count from N=50 to N=10 (ablation) | SAEMark · English | F1: N=10 achieves 98.0% F1 | Hurt |
| Reduce candidate count from N=50 to N=5 (ablation) | SAEMark · English | F1: N=5 attains 86.8% F1 | Hurt |
| Remove background feature masking (ablation) | SAEMark · Watermark detection | AUC: almost 1.0 -> 0.85 | Hurt |
| Preserve content structure during word deletion rather than disrupt it | SAEMark under word deletion attack at ε = 0.05 · Word deletion attack | AUC: 0.901 -> 0.949 | Helped |
| Preserve content structure during word deletion rather than disrupt it | SAEMark under word deletion attack at ε = 0.1 · Word deletion attack | AUC: 0.825 -> 0.858 | Helped |
Surprising findings
- On code (MBPP), SAEMark's F1 of 66.3% is below KGW's accuracy in terms of Acc (65.4) only marginally, but its recall is far higher than the other multi-bit method; code remains the hardest domain.
- Theoretical per-unit bounds (N=50 needed for >99%) are far more conservative than empirical results (N=5 gives 86.8% F1).
Limitations
- Relies on quality of pretrained SAE features and requires an anchor model with an SAE.
- Detection needs open-ended, sufficiently long text; short outputs such as multiple-choice answers carry too little signal.
- Does not claim cryptographic unforgeability when keys are known; higher compute cost from sampling N candidates per unit.
Experimental setup
Datasets: C4 LCSTS MBPP PandaLM BIGGen-Bench
Benchmarks: C4 LCSTS MBPP PandaLM BIGGen-Bench
Metrics: Accuracy Recall F1 at 1% FPR win-rate Likert score AUC
Baselines: KGW EXP UPV Unigram DIP Unbiased SynthID SWEET Waterfall
Builds on: KGW Waterfall MarkLLM EXP
Closest papers by meaning
- PRO: Enabling Precise and Robust Text Watermark for Open-Source LLMs (NeurIPS 2026 · similarity 0.91)
- Black-Box Detection of Language Model Watermarks (ICLR 2025 · similarity 0.90)
- An End-to-End Model for Logits-Based Large Language Models Watermarking (ICML 2025 · similarity 0.90)
- Theoretically Grounded Framework for LLM Watermarking: A Distribution-Adaptive Approach (NeurIPS 2025 · similarity 0.90)
- BiMark: Unbiased Multilayer Watermarking for Large Language Models (ICML 2025 · similarity 0.90)
- TextSeal: A Localized LLM Watermark for Provenance & Distillation Protection (NeurIPS 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.