Research map / Large language models / Language, safety and interpretability
Ward: Provable RAG Dataset Inference via LLM Watermarks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Formalizes black-box RAG dataset inference (RAG-DI), introduces the FARAD fact-redundant fictional article dataset, and proposes WARD, which watermarks a data owner's documents and aggregates watermark p-values from black-box RAG queries for statistically guaranteed dataset-level detection.
Problem
Detecting unauthorized use of a data owner's document collection in a RAG system's retrieval corpus via black-box queries, with statistical guarantees, under realistic conditions including fact redundancy and no training-data contamination.
Approach
Proactively watermark each document with red-green LLM watermarks, query the RAG system with content-related questions, and combine the resulting responses' green-token counts into a joint p-value; the watermark signal propagates through retrieval into responses, giving provable Type-1 error control without training or adaptation.
Key results
- WARD achieves 100% accuracy across all settings, models, and system prompts on FARAD Hard, while all baselines fail to perform consistently in the fact-redundant setting.
- WARD reaches perfect accuracy for at most 80 documents on FARAD Hard, with total API cost below $2 per LLM.
- FACTS obtains perfect results on FARAD Easy but is unreliable on FARAD Hard, and defensive system prompts cause IBM and SIB baselines to produce many false negatives.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use watermark-based Ward instead of the strongest baseline | RAG dataset inference · FARAD Hard | accuracy | Helped |
| Use Facts instead of other baselines | RAG dataset inference · FARAD Easy | accuracy | Helped |
| Add a defense system prompt | IBM and SIB · FARAD Easy | false negatives | Hurt |
| Increase the number of queried documents | Ward · FARAD Hard | accuracy | Helped |
| Increase the number of queried documents | SIB · FARAD Hard | accuracy | Mixed |
| Increase watermark strength δ above 4.5 (ablation) | Ward · FARAD Hard | query efficiency | Hurt |
| Use watermark strength δ in [3.5,4.5] (ablation) | Ward · FARAD Hard | cumulative accuracy | Helped |
| Increase watermark context size from h=2 to h=4 (ablation) | Ward · FARAD Hard | accuracy: 100\% -> 80\% | Hurt |
| Increase the number of retrieved documents k (ablation) | Ward · FARAD Hard | accuracy | No clear effect |
| Ask multiple questions per document (ablation) | Ward · FARAD Hard | query efficiency | Mixed |
| Apply MemFree decoding to prevent n-gram overlap (ablation) | Ward · FARAD Hard | accuracy | No clear effect |
| Use an end-to-end RAG system instead of perfect retrieval (ablation) | Ward · FARAD Easy and FARAD Hard | accuracy | No clear effect |
Surprising findings
- The simple FACTS baseline outperforms the existing RAG MIA baselines on the non-redundant Easy setting, suggesting traditional non-redundant datasets understate task difficulty.
- Watermark signal persists through the RAG pipeline even under MemFree decoding and defensive prompts that suppress verbatim regurgitation.
Limitations
- Evaluation uses a perfect (oracle-like) retriever in main experiments, with an end-to-end retrieval study only as a check.
- Dataset is LLM-written fictional articles rather than human-written, limiting realism; only 1,000 of 3,391 test groups were generated.
- Watermarking requires the data owner to paraphrase and watermark their own documents before publication, and the method assumes watermarks are not leaked into RAG responses by queries.
Experimental setup
Datasets: RepliQA EnronEmails HealthcareMagic
Benchmarks: FARAD Easy FARAD Hard
Metrics: accuracy p-value true positive rate false positive rate text quality score
Baselines: IBM-style RAG MIA (direct prompting) SIB-style RAG MIA (similarity and perplexity) FACTS
Builds on: red-green watermarks RAG membership inference attacks PositionPRF MemFree decoding RepliQA
Closest papers by meaning
- Follow My Instruction and Spill the Beans: Scalable Data Extraction from Retrieval-Augmented Generation Systems (ICLR 2025 · similarity 0.86)
- Worse than Zero-shot? A Fact-Checking Dataset for Evaluating the Robustness of RAG Against Misleading Retrievals (NeurIPS 2025 · similarity 0.86)
- STAMP Your Content: Proving Dataset Membership via Watermarked Rephrasings (ICML 2025 · similarity 0.85)
- SeCon-RAG: A Two-Stage Semantic Filtering and Conflict-Free Framework for Trustworthy RAG (NeurIPS 2025 · similarity 0.84)
- Combating Data Laundering in LLM Training (NeurIPS 2026 · similarity 0.84)
- Unlocking Post-hoc Dataset Inference with Synthetic Data (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.