Research map / Large language models / Retrieval and knowledge
Bridging Context Gaps: Leveraging Coreference Resolution for Long Contextual Understanding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LQCA, which resolves coreferences over sliding sub-documents with Maverick, merges them into document-level clusters via mention-distance graphs, replaces mentions with representative text, and answers questions with the cleaned context.
Problem
LLMs struggle to understand long contexts and answer questions accurately because of ambiguous references and coreference relations spread across lengthy texts.
Approach
Partition long documents into sub-documents, run a coreference resolver on each, merge cross-partition mention relations via a distance score and graph clustering, pick a representative non-pronoun mention per cluster, and substitute mentions in the text before prompting the LLM.
Key results
- LQCA improves GPT-4o by an average of +3.61% over vanilla prompting across the nine evaluated sub-tasks.
- On OpenAI-o1-mini, LQCA gives an average +3.18% gain over other methods and reaches 67.78 average score versus 64.60 for vanilla.
- Mid-document accuracy in the [0.4, 0.6] interval on Coursera rises by +6.1% for GPT-4o with coreference resolution.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply LQCA instead of the strongest listed baseline | OpenAI-o1-mini · LooGLE, L-Eval, and LongBench (Avg.) | average score: 64.60 -> 67.78 | Helped |
| apply LQCA instead of the strongest listed baseline | GPT-4o · LooGLE, L-Eval, and LongBench (Avg.) | average score: 61.21 -> 63.76 | Helped |
| apply LQCA instead of the strongest listed baseline | GPT-4o-mini · LooGLE, L-Eval, and LongBench (Avg.) | average score: 58.19 -> 61.17 | Helped |
| apply LQCA instead of the strongest listed baseline | Llama-3-8b · LooGLE, L-Eval, and LongBench (Avg.) | average score: 38.46 -> 39.37 | Helped |
| apply LQCA instead of the strongest listed baseline | Qwen2-7b · LooGLE, L-Eval, and LongBench (Avg.) | average score: 37.17 -> 38.44 | Helped |
| randomly replace 5% of mentions (ablation) | LQCA with GPT-4o · LooGLE-arxiv-paper-abstract | Rouge-L: 34.53 -> 28.54 | Hurt |
| randomly replace 5% of mentions (ablation) | LQCA with GPT-4o · L-Eval-TOEFL | accuracy: 91.03 -> 82.51 | Hurt |
| randomly replace 5% of mentions (ablation) | LQCA with GPT-4o · L-Eval-Coursera | accuracy: 85.12 -> 81.35 | Hurt |
| randomly replace 5% of mentions (ablation) | LQCA with GPT-4o · LongBench-HotpotQA | F1: 71.66 -> 68.77 | Hurt |
| randomly replace 5% of mentions (ablation) | LQCA with GPT-4o · LongBench-2WikiMHQA | F1: 53.62 -> 47.23 | Hurt |
| randomly replace 5% of mentions (ablation) | LQCA with OpenAI-o1-mini · LooGLE-arxiv-paper-abstract | Rouge-L: 40.65 -> 32.51 | Hurt |
| randomly replace 5% of mentions (ablation) | LQCA with OpenAI-o1-mini · L-Eval-TOEFL | accuracy: 93.71 -> 88.22 | Hurt |
Surprising findings
- LLM-based coreference resolution performed poorly for question answering, underperforming the rule-based Maverick pipeline.
- Retrieval augmentation (LQCA-w/ RAG) outperformed full LQCA on some metrics for smaller models.
Limitations
- Coreference errors on ambiguous or complex contexts can yield wrong mention replacements.
- Benefit may diminish for shorter contexts where processing overhead outweighs gains.
- Reliance on pre-defined mention distance thresholds may limit adaptability across domains.
Experimental setup
Datasets: LooGLE L-Eval LongBench (HotpotQA, 2WikiMultihopQA, MuSiQue)
Benchmarks: LooGLE L-Eval LongBench
Metrics: Rouge-L accuracy F1
Baselines: Vanilla LM Zero-shot CoT RAG with self-Corpus RECOMP with self-Corpus
Builds on: Maverick DeBERTa-v3 Zero-shot chain-of-thought RAG RECOMP
Closest papers by meaning
- Retrieval meets Long Context Large Language Models (ICLR 2024 · similarity 0.88)
- Training with “Paraphrasing the Original Text” Teaches LLM to Better Retrieve in Long-Context Tasks (AAAI 2025 · similarity 0.87)
- NovelQA: Benchmarking Question Answering on Documents Exceeding 200K Tokens (ICLR 2025 · similarity 0.87)
- SuRe: Summarizing Retrievals using Answer Candidates for Open-domain QA of LLMs (ICLR 2024 · similarity 0.87)
- Making Retrieval-Augmented Language Models Robust to Irrelevant Context (ICLR 2024 · similarity 0.87)
- RepLiQA: A Question-Answering Dataset for Benchmarking LLMs on Unseen Reference Content (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.