Research map / Large language models / Retrieval and knowledge
FaithEval: Can Your Language Model Stay Faithful to Context, Even If "The Moon is Made of Marshmallows"
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces FaithEval, a 4.9K-sample benchmark testing LLM contextual faithfulness via unanswerable, inconsistent, and counterfactual contexts, finding that even top models struggle and larger models are not reliably more faithful.
Problem
Existing hallucination benchmarks rarely measure faithfulness to noisy, long contexts (missing, conflicting, or counterfactual evidence) as used in RAG, and often conflate factuality with faithfulness.
Approach
Builds three context-perturbation tasks from existing contextual QA datasets and ARC-Challenge, generating modified contexts with GPT-4o and validating them with an LLM judge (gpt-4o-mini), string matching, and human annotators (Mechanical Turk) to create a benchmark of 4.9K contextual QA pairs.
Key results
- Phi-3-medium-128k-instruct reaches 75.8% on original context but only 7.4% on unanswerable context.
- GPT-4o achieves 96.3% closed-book on counterfactual QA but only 47.5% with counterfactual context.
- Phi-3 series averages 5.8% on inconsistent context while GPT-4 series averages 89.35%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Provide counterfactual evidence in the context instead of evaluating closed-book | GPT-4o · Counterfactual Context | accuracy: 96.3% -> 47.5% | Hurt |
| Add the task-specific conflict instruction (ablation) | Claude 3.5 Sonnet · Normal context (AVG) | accuracy: 0.90 -> 0.85 | Hurt |
| Add the task-specific conflict instruction (ablation) | GPT-4o · Normal context (AVG) | accuracy: 0.85 -> 0.83 | Hurt |
| Increase model size from 8B to 70B | Llama-3.1 · Unanswerable Context | accuracy: 10.3% | Helped |
| Use chain-of-thought prompting instead of direct-answer prompting (ablation) | LLMs · Unanswerable and Inconsistent Context | accuracy | Helped |
| Use sampling-based decoding (temperature 0.3, top-p 0.9) instead of greedy decoding (ablation) | LLMs · Counterfactual Context | accuracy | Helped |
| Allow non-strict matching of semantically similar abstention phrases instead of strict phrase matching (ablation) | LLMs · Unanswerable Context | accuracy | No clear effect |
| Present the new context alone rather than concatenating it with the original context (ablation) | Command R · Inconsistent Context | accuracy: 81% -> 88% | Helped |
| Use the unanswerable context instead of the original context | Phi-3-medium-128k-instruct · Unanswerable Context | accuracy: 75.8% -> 7.4% | Hurt |
| Use the inconsistent context instead of the original context | Phi-3 series · Inconsistent Context | accuracy | Hurt |
Surprising findings
- Larger models do not necessarily exhibit improved faithfulness; e.g., GPT-4o and Llama-3-70B-Instruct do not reliably outperform smaller models on faithfulness tasks.
- Adding a task-specific conflict instruction degrades accuracy on normal, answerable contexts (sycophancy-like effect), with Claude 3.5 Sonnet dropping about 5% on average.
Limitations
- Inconsistent and unanswerable contexts were generated by GPT-4o, which may introduce generator-specific artifacts.
- Counterfactual task is built only from ARC-Challenge multiple-choice questions and validated by string matching rather than full human review.
Experimental setup
Datasets: SQuAD NewsQA TriviaQA NaturalQuestions SearchQA HotpotQA BioASQ DROP RACE TextbookQA ARC-Challenge
Benchmarks: FaithEval
Metrics: accuracy (strict and non-strict matching)
Baselines: Phi-3-mini-128k-instruct Phi-3-medium-128k-instruct Phi-3.5-mini-instruct Llama-3-8B-Instruct Llama-3.1-8B-Instruct Llama-3-70B-Instruct Llama-3.1-70B-Instruct Mistral-7B-Instruct-v0.3 Mistral-Nemo-Instruct-2407 gemma-2-9b-it gemma-2-27b-it GPT-3.5 Turbo
Builds on: chain-of-thought
Closest papers by meaning
- To Trust or Not to Trust? Enhancing Large Language Models' Situated Faithfulness to External Contexts (ICLR 2025 · similarity 0.91)
- Is Factuality Enhancement a Free Lunch For LLMs? Better Factuality Can Lead to Worse Context-Faithfulness (ICLR 2025 · similarity 0.88)
- ParamMute: Suppressing Knowledge-Critical FFNs for Faithful Retrieval-Augmented Generation (NeurIPS 2025 · similarity 0.88)
- Walk the Talk? Measuring the Faithfulness of Large Language Model Explanations (ICLR 2025 · similarity 0.88)
- Aligning Large Language Models for Faithful Integrity Against Opposing Argument (AAAI 2025 · similarity 0.87)
- HonestLLM: Toward an Honest and Helpful Large Language Model (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.