Research map / Large language models / Retrieval and knowledge
The Narrative Gap: Can LLMs help us Navigate Diverse Narratives Across Languages?
Nikhil Sharma, Kelly Marchisio, Kenton Murray, Ziang Xiao
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces a benchmark for testing whether LLMs can navigate diverse narratives about the same event across languages and regions.
Problem
No existing benchmark evaluates whether LLMs can navigate the diverse narratives that develop around the same event across languages and regions, which matters for information seeking about contested topics.
Approach
Argues that supporting diverse information seeking requires four core information navigation capabilities, and builds a benchmark to evaluate LLMs on narrative navigation across languages (details beyond the abstract are not stated).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add an explicit diversity request (diverse prompt) to the query (ablation) | 14 LLMs on NarrativeBench · NarrativeBench narrative recall | narrative recall (%): 44.3 -> 49.2 | Improved |
| add an explicit diversity request (diverse prompt) (ablation) | Gemini 2.5 Pro · NarrativeBench narrative recall | narrative recall (%): 70.6 -> 74.9 | Improved |
| ask for a specific narrative (inclusion) rather than excluding one | 14 LLMs · NarrativeBench NI vs NE | narrative F1 (%): NE 49.3 vs NI 19.9 | Hurt |
| use confirmatory instead of neutral query framing (ablation) | 14 LLMs · NarrativeBench BP/DP recall | absolute recall gap (pp): mean gap 3.23 pp; range 0.32 to 10.09 | Mixed |
| query in non-English language instead of English | 14 LLMs · NarrativeBench all tasks | mean absolute gap vs English (pp): 8.74 pp query-language; 7.51 pp target narrative language | Hurt |
| query in the same language as the source articles (ablation) | source-language omission · NarrativeBench BP and DP | source-language omission rate (%): 50.4 -> 41.3 | Improved |
| enable medium reasoning (ablation) | GPT-5.4-mini · NarrativeBench NI | narrative F1 (%): 28.11 -> 39.37 | Improved |
| enable medium reasoning (ablation) | GPT-5.4-mini · NarrativeBench NE, MNI, BP, DP | F1/recall (pp): decrease of 7.38-15.94 pp | Hurt |
| minimum thinking settings and caps (ablation) | Gemini Flash · NarrativeBench five tasks | score (pp): decrease of 8.09-10.59 pp | Hurt |
| higher translation quality (COMETKiwi) (ablation) | benchmark task scores · NarrativeBench, 8 languages | task score: +0.17 pp per 0.01 COMETKiwi, CI [-0.15, 0.49] | No clear effect |
| longer input context (per doubling) (ablation) | task scores · NarrativeBench supplementary 1,720 cases | task score (pp): +4.3 pp, 95% CI [-0.7, 9.3] | No clear effect |
| share of post-knowledge-cutoff articles in context (ablation) | nine models · NarrativeBench | recall (pp): +4.77 pp, SE 4.84 | No clear effect |
Closest papers by meaning
- More Is Not More: What Matters for Diversity in LLM Opinions? (NeurIPS 2026 · similarity 0.86)
- Evaluating Whether LLMs Can Reliably Connect the DOTs? (NeurIPS 2026 · similarity 0.86)
- CulturePark: Boosting Cross-cultural Understanding in Large Language Models (NeurIPS 2024 · similarity 0.85)
- BLEnD: A Benchmark for LLMs on Everyday Knowledge in Diverse Cultures and Languages (NeurIPS 2024 · similarity 0.85)
- StereoTales: A Multilingual Framework for Open-Ended Stereotype Discovery in LLMs (NeurIPS 2026 · similarity 0.84)
- Diversity as a Reward: Fine-Tuning LLMs on a Mixture of Domain-Undetermined Data (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
