Research map / Adversarial robustness and security / Attacks on text and graphs
ReliabilityRAG: Effective and Provably Robust Defense for RAG-based Web-Search
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ReliabilityRAG, which filters retrieved documents by finding a maximum independent set over an NLI-derived contradiction graph (with rank-based tie-breaking) plus a weighted sample-and-aggregate variant for large retrieval sets, evaluated against prompt injection and corpus poisoning in RAG QA and long-form biography generation.
Problem
RAG systems for web search are vulnerable to corpus poisoning and prompt injection in retrieved documents; existing defenses like RobustRAG lose benign accuracy and struggle on long-form generation, and largely ignore document reliability signals such as rank or weight.
Approach
Treat a "consistent majority" as the maximum independent set in a contradiction graph whose edges come from NLI judgments over isolated per-document LLM answers, break ties toward higher-ranked documents, and scale via weighted sampling of small contexts followed by MIS aggregation.
Key results
- On NQ with k=10 under prompt injection at position 10, MIS with Mistral-7B reaches 58% vs 13.6% for vanilla RAG and 44% for RobustRAG.
- On Bio with Llama3.2-3B, MIS scores 73 benign and 72.1 at position 10, while RobustRAG (Keyword) scores 56 benign and 51.9 at position 10.
- Sampling + MIS with GPT-4o-mini achieves the highest benign accuracy on all four datasets among compared methods (e.g., 77% on RQA at k=50).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use rank-aware MIS filtering instead of RobustRAG | Mistral-7B RAG · RQA, prompt injection at position 1 | accuracy (%): 53 -> 68 | Helped |
| Use rank-aware MIS filtering instead of RobustRAG | Mistral-7B RAG · NQ, prompt injection at position 10 | accuracy (%): 44 -> 58 | Helped |
| Use rank-aware MIS filtering instead of RobustRAG | Mistral-7B RAG · Bio, prompt injection at position 10 | LLM-Judge score: 57.1 -> 71.5 | Helped |
| Use rank-aware MIS filtering instead of RobustRAG | Mistral-7B RAG · RQA, benign | accuracy (%): 56 -> 70 | Helped |
| Use weighted Sampling + MIS rather than RobustRAG | Mistral-7B RAG, k=50 · RQA, prompt injection at position 25 | accuracy (%): 51 -> 68 | Helped |
| Use weighted Sampling + MIS rather than RobustRAG | Mistral-7B RAG, k=50 · NQ, prompt injection at position 50 | accuracy (%): 47 -> 60 | Helped |
| Use weighted Sampling + MIS rather than RobustRAG | Mistral-7B RAG, k=50 · Bio, prompt injection at position 25 | LLM-Judge score: 59.8 -> 69.7 | Helped |
| Attack the lower-ranked rather than highest-ranked document | MIS with Mistral-7B · RQA | accuracy (%): 68 -> 60 | Hurt |
| Attack a lower-ranked document rather than the highest-ranked document | Sampling + MIS with Mistral-7B · RQA, k=50 | accuracy (%): 54 -> 72 | Helped |
| Increase the number of attacked documents in a suffix of retrieved passages | Sampling + MIS · NQ, k=50 | accuracy (%) | Hurt |
Surprising findings
- MIS accuracy under attack is often higher when the attack targets lower-ranked (Position 10) than top-ranked (Position 1) documents, which the authors attribute partly to Mistral-7B favoring content at the end of context.
- Robustness-focused baseline RobustRAG (Keyword) degrades sharply on long-form Bio generation, while MIS retains utility.
Limitations
- Robustness guarantees rely on assumed bounded NLI error rates and bounded corruption (k' <= k/5), which the authors note are not strongly supported by the NLI accuracy numbers.
- Relies on benign documents being relevant; an 'I don't know' filter is used to handle noisy benign documents, and its effect is not ablated.
- Sampling+MIS uses only ranks in the final aggregation, not weights, potentially losing information.
Experimental setup
Datasets: RealtimeQA Natural Questions TriviaQA Biography generation dataset Google Search results
Benchmarks: RealtimeQA Natural Questions TriviaQA Biography generation
Metrics: accuracy LLM-judge score attack success rate
Baselines: Vanilla RAG RobustRAG (Keyword) AstuteRAG InstructRAG
Builds on: RobustRAG Maximum Independent Set weighted sample and aggregate DeBERTa NLI
Closest papers by meaning
- On the Vulnerability of Applying Retrieval-Augmented Generation within Knowledge-Intensive Application Domains (ICML 2025 · similarity 0.89)
- Worse than Zero-shot? A Fact-Checking Dataset for Evaluating the Robustness of RAG Against Misleading Retrievals (NeurIPS 2025 · similarity 0.87)
- RAGGED: Towards Informed Design of Scalable and Stable RAG Systems (ICML 2025 · similarity 0.86)
- Follow My Instruction and Spill the Beans: Scalable Data Extraction from Retrieval-Augmented Generation Systems (ICLR 2025 · similarity 0.86)
- SeCon-RAG: A Two-Stage Semantic Filtering and Conflict-Free Framework for Trustworthy RAG (NeurIPS 2025 · similarity 0.86)
- Closing the Gap: Achieving Better Accuracy-Robustness Tradeoffs against Query-Based Attacks (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.