Research map / Large language models / Retrieval and knowledge
Measuring and Enhancing Trustworthiness of LLMs in RAG through Grounded Attributions and Learning to Refuse
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces TRUST-Score, a metric for LLM groundedness in RAG (refusals, answer correctness, citation groundedness), and TRUST-Align, a DPO-based alignment on a 19K-sample dataset of preferred/unpreferred responses that improves LLaMA, Qwen, and Phi models over baselines on ASQA, QAMPARI, and ELI5.
Problem
LLMs in RAG often rely on parametric knowledge, refuse or over-answer inappropriately, and produce poorly grounded citations; existing RAG metrics conflate retriever quality with LLM quality and inflate answer correctness via parametric knowledge.
Approach
A holistic trustworthiness metric isolating the LLM within RAG (grounded refusals, answer correctness calibrated to retrieved documents, and citation groundedness), plus an alignment dataset of hallucination-targeted preference pairs used with DPO to align models.
Key results
- LLaMA-3-8b aligned with DPO outperforms FRONT by 12.56 (ASQA), 36.04 (QAMPARI), and 17.69 (ELI5) TRUST-Score points.
- DPO-aligned models outperform baselines on TRUST-Score in 26 of 27 model/dataset configurations.
- On ExpertQA, DPO-aligned LLaMA-3-8b reaches TRUST-Score 54.85 versus 54.69 for GPT-4.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply Trust-Align DPO alignment rather than FRONT | LLaMA-3-8b · ASQA | TRUST: 56.67 -> 69.23 | Helped |
| Apply Trust-Align DPO alignment rather than FRONT | LLaMA-3-8b · QAMPARI | TRUST: 18.91 -> 54.95 | Helped |
| Apply Trust-Align DPO alignment rather than FRONT | LLaMA-3-8b · ELI5 | TRUST: 27.18 -> 44.87 | Helped |
| Add augmented instructions to the alignment data (ablation) | Trust-Align on LLaMA-2-7b · ASQA | TRUST: 66.01 -> 67.51 | Helped |
| Add answer-related hallucination data (ablation) | Trust-Align on LLaMA-2-7b · ASQA | TRUST: 66.06 -> 67.51 | Helped |
| Add citation-related hallucination data (ablation) | Trust-Align on LLaMA-2-7b · ASQA | TRUST: 65.98 -> 67.51 | Helped |
| Add refusal-related hallucination data (ablation) | Trust-Align on LLaMA-2-7b · QAMPARI | F1_GR: 71.35 -> 71.83 | Helped |
| Add unanswerable questions and refusal responses to training (ablation) | Trust-Align on LLaMA-2-7b · ASQA | TRUST: 56.10 -> 67.51 | Helped |
| Replace the Trust-Align preference-data construction with GPT-4-as-critic (ablation) | LLaMA-2-7b · ELI5 | TRUST: 40.63 -> 44.75 | Hurt |
| Use DPO alignment rather than SFT | LLaMA-3.2-3b · ASQA | TRUST: 63.44 -> 70.14 | Helped |
| Use DPO alignment rather than SFT | LLaMA-3.2-3b · QAMPARI | TRUST: 45.45 -> 48.54 | Helped |
| Use DPO alignment rather than SFT | LLaMA-3.2-3b · ELI5 | TRUST: 43.95 -> 45.66 | Helped |
Surprising findings
- Prompting methods such as in-context learning fail to adapt LLMs to the RAG task, and closed-source models like GPT-4 and Claude-3.5-Sonnet rely heavily on parametric knowledge.
- Answer correctness (F1_AC) improvements from alignment are mixed on ASQA and ELI5 (2/9 and 5/9 models), because aligned models answer fewer questions.
Limitations
- F1_AC can be inflated by over-responsive models, so answer correctness is mixed and gameable.
- Answerability and entailment rely on TRUE T5-11B NLI and substring matching, which may be imperfect.
- Negative responses come from a LLaMA-2-7b SFT model, and the dataset relies on GPT-4 generated gold responses.
Experimental setup
Datasets: ASQA QAMPARI ELI5 ExpertQA
Benchmarks: ASQA QAMPARI ELI5 ExpertQA
Metrics: TRUST-Score F1_AC F1_GR F1_GC answered ratio
Baselines: ICL PostCite PostAttr Self-RAG FRONT GPT-4 as critic
Builds on: direct preference optimization FRONT ASQA ExpertQA
Closest papers by meaning
- ChatQA 2: Bridging the Gap to Proprietary LLMs in Long Context and RAG Capabilities (ICLR 2025 · similarity 0.88)
- Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAG (ICLR 2025 · similarity 0.88)
- Toward Verifiable Instruction-Following Alignment for Retrieval Augmented Generation (AAAI 2025 · similarity 0.88)
- Position: TrustLLM: Trustworthiness in Large Language Models (ICML 2024 · similarity 0.87)
- On Evaluating LLM Alignment by Evaluating LLMs as Judges (NeurIPS 2025 · similarity 0.87)
- Speculative RAG: Enhancing Retrieval Augmented Generation through Drafting (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.