Research map / Large language models / Retrieval and knowledge
ERBench: An Entity-Relationship based Automatically Verifiable Hallucination Benchmark for Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ERBench, which converts relational databases into LLM hallucination benchmarks using functional dependencies and foreign key constraints to auto-verify both answers and rationales, evaluated on six LLMs across five domains.
Problem
Existing automatic LLM benchmarks built from knowledge graphs or QA datasets use simple triple-based questions and only check final answers, so they cannot verify reasoning rationales or evaluate complex multi-hop factual knowledge.
Approach
Use integrity constraints of relational databases (functional dependencies to pick critical keywords that must appear in the rationale, and foreign key constraints to join relations for arbitrarily long multi-hop questions) to automatically generate binary and multiple-choice questions and verify both the answer and the rationale.
Key results
- GPT-4 achieves the best answer and rationale accuracy on several single-hop settings, e.g., 0.97 answer accuracy on Movie multiple-choice, while rationale accuracy is generally lower than answer accuracy across models.
- ERBench's automatic rationale verification matches manual verification with 92.8% to 97.4% accuracy across the six evaluated LLMs.
- Fine-tuning GPT-3.5 on 3,000 Soccer entities raises Movie binary answer accuracy from 0.85 to 0.88 and Music from 0.58 to 0.93 (negated and other settings vary).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use multiple-choice instead of binary basic single-hop questions | LLMs evaluated with ERBench · Movie, Soccer, Airport, Music, and Book | answer accuracy | Helped |
| Add Chain-of-Thought prompting (ablation) | LLMs answering multi-hop questions · Movie & Director and Soccer & Olympic | answer accuracy: .59 -> .80; .46 -> .79 | Mixed |
| Add few-shot demonstrations (8 for binary; 2–4 for multiple-choice) (ablation) | LLMs answering single-hop questions · Single-hop questions across domains | performance | Helped |
| Add Wikipedia background passages through knowledge augmentation (ablation) | LLMs answering ERBench questions · Single-hop questions | performance | Hurt |
| Replace title and club text values with movie posters and soccer club logos (ablation) | GPT-4V and Gemini-Pro-Vision · Movie and Soccer multimodal questions | performance | Helped |
| Fine-tune GPT-3.5 for 2 epochs on 3,000 Soccer entities (ablation) | GPT-3.5 · Movie, Soccer, Airport, Music, and Book | answer accuracy: .85 -> .88; .35 -> .92; .13 -> .94; .58 -> .93; .77 -> .94 | Helped |
| Fine-tune GPT-3.5 for 2 epochs on Movie, Soccer, Music, and Book (ablation) | GPT-3.5 · Airport and Music | answer accuracy: .13 -> .51; .58 -> .20 | Mixed |
| Evaluate only on entities known to all LLMs, excluding long-tail entities | LLMs evaluated with ERBench · Single-hop questions across five datasets | performance | Helped |
| Evaluate on all questions regardless of model knowledge | LLMs evaluated with ERBench · Single-hop questions across five datasets | performance | Hurt |
Surprising findings
- Adding Wikipedia knowledge augmentation often degraded LLM performance because the passages may mislead the model.
- Fine-tuning on more datasets did not necessarily improve performance over fine-tuning on Soccer alone.
Limitations
- Rationale verification only checks for critical keywords, not the entire rationale content.
- Entity resolution relies on heuristics and string matching, which can miss or mis-match entities.
- Results depend on the LLM's knowledge of entities, requiring filtering by known entities.
Experimental setup
Datasets: Movie Soccer Airport Music Book Director Club Olympic
Benchmarks: ERBench Movie ERBench Soccer ERBench Airport ERBench Music ERBench Book ERBench multi-hop (Movie & Director, Soccer & Olympic) ERBench multimodal
Metrics: answer accuracy rationale accuracy answer-rationale accuracy hallucination rate
Baselines: GPT-3.5 GPT-4 Llama2-70B-Chat Gemini-Pro Claude-3-Sonnet Mistral-7B-Instruct GPT-4V Gemini-Pro-Vision GPT-Judge
Builds on: Head-to-Tail chain-of-thought prompting few-shot prompting
Closest papers by meaning
- metabench - A Sparse Benchmark of Reasoning and Knowledge in Large Language Models (ICLR 2025 · similarity 0.89)
- Is There No Such Thing as a Bad Question? H4R: HalluciBot for Ratiocination, Rewriting, Ranking, and Routing (AAAI 2025 · similarity 0.88)
- Logical Consistency of Large Language Models in Fact-Checking (ICLR 2025 · similarity 0.88)
- AutoBencher: Towards Declarative Benchmark Construction (ICLR 2025 · similarity 0.87)
- TreeEval: Benchmark-Free Evaluation of Large Language Models through Tree Planning (AAAI 2025 · similarity 0.87)
- ThinkBench: Dynamic Out-of-Distribution Evaluation for Robust LLM Reasoning (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.