Research map / Adversarial robustness and security / Attacks on text and graphs
RLCracker: Evaluating the Worst-Case Vulnerability of LLM Watermarks with Adaptive RL Attacks
Hanbo Huang, Yiran Zhang, Hao Zheng, Xuan Gong, Yihan Li, Lin Liu, Zhuotao Liu, Shiyu Liang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add role-assignment system prompt to paraphrase attack | Qwen3-4B paraphraser · EWD 500 tokens | ESR (%): 39.3 -> 54.3 | Improved |
| add role-assignment system prompt to paraphrase attack | Qwen3-4B paraphraser · KGW selfhash 500 tokens | ESR (%): 70.0 -> 79.3 | Improved |
| prompt paraphraser to reason before rewriting (Think) | Qwen3-4B paraphraser · EWD 500 tokens | ESR (%): 39.3 -> 76.0 | Improved |
| prompt paraphraser to reason before rewriting (Think) | Qwen3-4B paraphraser · KGW selfhash 1500 tokens | ESR (%): 36.0 -> 78.0 | Improved |
| add more enforced reasoning steps at test time | Qwen3-8B paraphraser · SIR and PF 500 tokens | ESR (%) | Improved |
| train paraphraser with RLCracker instead of GPT-4o prompting | Qwen2.5-3B-Instruct · Unigram 1500 tokens | ESR (%): 6.75 -> 98.5 | Improved |
| train with RLCracker instead of SIRA | Qwen2.5-3B-Instruct · Unigram 1500 tokens | P-SP score: 0.47 -> 0.92 | Improved |
| apply RLCracker to a Think+SysP baseline on Unigram 500 tokens | Qwen3-4B · Unigram 500 tokens | ESR (%): 38.8 -> 73.5 | Improved |
| increase training samples from 50 to 100 (ablation) | RLCracker Qwen3-0.6B · EWD 250 tokens | ESR (%): 82.5 -> 87.8 | Improved |
| increase training samples from 100 to 200 (ablation) | RLCracker Qwen3-0.6B · EWD 500 tokens | ESR (%): 91.5 -> 92.0 | No clear effect |
| increase training token length per sample from 250 to 500 (ablation) | RLCracker Qwen3-0.6B · EWD (50 samples) | ESR (%): 82.5 -> 86.5 | Improved |
| train on mixed-key watermarked data instead of single key (ablation) | RLCracker Qwen3-0.6B · EWD 500 tokens | ESR (%): 91.5 -> 88.8 | Hurt |
Closest papers by meaning
- Optimizing Adaptive Attacks against Watermarks for Language Models (ICML 2025 · similarity 0.92)
- Theoretically Grounded Framework for LLM Watermarking: A Distribution-Adaptive Approach (NeurIPS 2025 · similarity 0.90)
- Watermarks in the Sand: Impossibility of Strong Watermarking for Language Models (ICML 2024 · similarity 0.89)
- On the Learnability of Watermarks for Language Models (ICLR 2024 · similarity 0.89)
- Adaptive Text Watermark for Large Language Models (ICML 2024 · similarity 0.89)
- On the Reliability of Watermarks for Large Language Models (ICLR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
