Research map / Large language models / Retrieval and knowledge

Coarse-to-Fine Highlighting: Reducing Knowledge Hallucination in Large Language Models

ICML 2024 · Large language models › Retrieval and knowledge · approach: Knowledge editing in LLMs · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes COFT, a training-free coarse-to-fine highlighting pipeline (recaller, scorer, selector) that bolds key entities in long reference contexts to reduce knowledge hallucination and improve long-context QA and reading comprehension with frozen LLMs.

Problem

Retrieval-augmented LLMs can hallucinate more when given long retrieved contexts, losing key information in lengthy reference texts.

Approach

Extract candidate key entities via a Wikidata knowledge graph and named entity recognition, score them by combining sentence-level TF-ISF with self-information from a small LM (Llama 7B), select a dynamic proportion using a length/informativeness threshold, and highlight the selected word, sentence, or paragraph units with markdown bold before passing the context to the LLM.

Key results

• On FELM with ChatGPT, COFT_w reaches 81.6 F1 on WK versus 25.2 for RALM and 9.1 for vanilla.
• On RACE-H with ChatGPT, COFT reaches 73.4 precision versus 65.6 for vanilla.
• On NQ with ChatGPT at 80% noise, COFT achieves 33.6 EM versus 25.9 for vanilla.

Measured effects reported in the paper

Surprising findings

• Chain-of-thought prompting often lowers performance on reading comprehension and hallucination benchmarks (e.g., ChatGPT RACE-H precision drops from 65.6 to 56.3).

Limitations

• Cannot update knowledge stored within LLMs.
• Relies on a knowledge graph (Wikidata) and an NER step for candidate entities.
• Highlighting uses only markdown bold; other formatting schemes left unexplored.

Experimental setup

Datasets: FELM RACE-M RACE-H Natural Questions TriviaQA WebQ Wikidata

Benchmarks: FELM RACE-M RACE-H Natural Questions TriviaQA WebQ

Metrics: F1 score precision recall EM F1

Baselines: Vanilla CoT RALM (BM25) CoVe CoN

Builds on: chain-of-note chain-of-verification TF-IDF FELM

Closest papers by meaning

• ReDeEP: Detecting Hallucination in Retrieval-Augmented Generation via Mechanistic Interpretability (ICLR 2025 · similarity 0.89)
• In-Context Sharpness as Alerts: An Inner Representation Perspective for Hallucination Mitigation (ICML 2024 · similarity 0.89)
• Zero-resource Hallucination Detection for Text Generation via Graph-based Contextual Knowledge Triples Modeling (AAAI 2025 · similarity 0.89)
• FINER: MLLMs Hallucinate under Fine-grained Negative Queries (CVPR 2026 · similarity 0.88)
• Teaching Language Models to Hallucinate Less with Synthetic Tasks (ICLR 2024 · similarity 0.88)
• ANAH-v2: Scaling Analytical Hallucination Annotation of Large Language Models (NeurIPS 2024 · similarity 0.88)