Research map / Large language models / Language, safety and interpretability
Do Automatic Factuality Metrics Measure Factuality? A Critical Evaluation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Stress-tests automatic summary factuality metrics (QuestEval, SummaC-Conv, UniEval, AlignScore, MiniCheck, GPT-4o-mini ChatGPT-DA) by difficulty-stratifying benchmarks with a shallow MLP, measuring sensitivity to factual corrections vs benign edits, gaming with innocuous phrases, and parametric-knowledge reliance.
Problem
Automatic factuality metrics for summarization may rely on superficial cues rather than measuring factual consistency, making them unreliable and gameable.
Approach
Use a shallow feature-based MLP probe to split benchmark examples into easy/medium/hard, then stress-test metrics via factual-correction vs benign-edit sensitivity, appended innocuous phrases, and counterfactual references to see what they actually capture.
Key results
- All metrics show substantial AUC drops from easy to hard examples; specialized models UniEval, MiniCheck and ChatGPT-DA are more robust on medium/hard cases.
- Constant gaming phrases raise SummaC-Conv, UniEval, AlignScore and MiniCheck scores by more than 0.2 absolute, and suffixes add 0.1-0.15, comparable to gains from factual corrections.
- GPT rates unsupported-but-factual summaries above supported ones in 3% of cases with counterfactual references versus 0.2% with factual references.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Append fixed, content-free phrases to summaries (ablation) | NLI and specialized factuality metrics · Summary manipulation experiment | factuality score: >0.2 points (absolute) | Helped |
| Append fixed phrases as suffixes to summaries (ablation) | Factuality metrics · Summary manipulation experiment | factuality score: 0.1–0.15 points | Helped |
| Append a random but consistent source sentence to an inconsistent summary (ablation) | Most factuality metrics except QuestEval and ChatGPT-DA · GenAudit summary pairs | factuality score | Helped |
| Correct factual errors in summaries (ablation) | QuestEval · GenAudit summary pairs | factuality score | No clear effect |
| Apply meaning-preserving negation (ablation) | MiniCheck and UniEval · GenAudit summary pairs | factuality score | Hurt |
| Make meaning-preserving benign edits to summaries (ablation) | ChatGPT-DA · GenAudit summary pairs | factuality score | No clear effect |
| Evaluate against counterfactual references that contradict parametric knowledge (ablation) | ChatGPT-DA · ConflictBank | supported-versus-unsupported score gap: p < 0.001 | Hurt |
| Evaluate against counterfactual references that contradict parametric knowledge (ablation) | ChatGPT-DA · ConflictBank | rate unsupported summaries score higher than supported summaries: 0.2% to 3.1% | Hurt |
| Assess summaries on examples that are hard for a shallow-feature classifier rather than easy examples | All automatic factuality metrics · Summary difficulty levels | AUC | Hurt |
| Use ChatGPT-DA instead of specialized metrics for benign-edit robustness and factual corrections | Factuality evaluation · GenAudit summary pairs | sensitivity to corrections and robustness to benign edits | Helped |
Surprising findings
- Constant non-content phrases such as 'the document discusses' boost scores of several specialized and NLI metrics more than genuine model improvements.
- SummaC-Conv shows no score increase for factually corrected summaries.
Limitations
- Benign edits generated by GPT-4 may occasionally introduce factual inconsistencies.
- Manipulations may produce out-of-distribution inputs, complicating interpretation.
- Study focuses on English-language metrics only.
Experimental setup
Datasets: AggreFact FacEval LLM-AggreFact GenAudit LLM-Dialogue SAMSum DialogSum ConflictBank XSUM CNNDM
Benchmarks: AggreFact FacEval LLM-AggreFact GenAudit LLM-Dialogue ConflictBank
Metrics: AUC score difference pairwise score shift
Baselines: QuestEval SummaC-Conv UniEval AlignScore MiniCheck
Builds on: SummaC QuestEval UniEval AlignScore MiniCheck
Closest papers by meaning
- Is Factuality Enhancement a Free Lunch For LLMs? Better Factuality Can Lead to Worse Context-Faithfulness (ICLR 2025 · similarity 0.89)
- Long-form factuality in large language models (NeurIPS 2024 · similarity 0.87)
- Fine-Tuning Language Models for Factuality (ICLR 2024 · similarity 0.87)
- The FACTS Leaderboard: A Comprehensive Benchmark for Large Language Model Factuality (NeurIPS 2026 · similarity 0.87)
- BooookScore: A systematic exploration of book-length summarization in the era of LLMs (ICLR 2024 · similarity 0.86)
- Trust, but Don’t Verify: Epistemic Blind Spots in LLM Source Evaluation (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.