Research map / Large language models / Retrieval and knowledge
A Human-Inspired Reading Agent with Gist Memory of Very Long Contexts
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ReadAgent, a prompting-only LLM agent that paginates long text, compresses pages into gist memories, and interactively looks up raw pages to answer questions, extending effective context 3.5-20x.
Problem
LLMs have limited context windows and degrade on long inputs, so they struggle to reason over very long documents such as books, movie scripts, and meeting transcripts.
Approach
Mimics human reading with a zero-shot prompting pipeline: the LLM chooses natural pause points to form pages, writes short gists for each page, and then decides which raw pages to look up from the gist memory when answering a task.
Key results
- On QuALITY dev, ReadAgent-S (look up 1-6 pages) reaches 87.17% accuracy vs 85.83% for full raw content, with ReadAgent-P (1-2 pages) at 86.16% and 72.17% compression.
- On NarrativeQA Gutenberg test, ReadAgent-P (1 page) improves LLM-Rating-2 and ROUGE-L over the best retrieval baseline, with about 20x effective context extension.
- On QMSum validation, ReadAgent-S reaches LLM-Rating-1 of 46.57% vs 40.81% for the best neural retrieval baseline.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ReadAgent-S with sequential page lookup instead of full raw text | PaLM 2-L · QuALITY dev | accuracy: 85.83% ± 0.19 -> 87.17% ± 0.18 | Helped |
| Use ReadAgent-P with one-page lookup instead of BM25 Top-4 | PaLM 2-L · NarrativeQA Gutenberg test | ROUGE-L: 0.197 -> 0.226 | Helped |
| Use ReadAgent-S with sequential lookup instead of neural retrieval | PaLM 2-L · QMSum validation | LLM Rating-1: 40.81% ± 0.52 -> 46.57% ± 0.87 | Helped |
| Replace prompt-based page selection with neural retrieval (ablation) | ReadAgent-P, QuALITY · QuALITY dev | accuracy: 84.13% -> 82.65% | Hurt |
| Use LLM-selected natural pause points instead of uniform-length pagination (ablation) | ReadAgent-P (1-5 pages) · QuALITY dev | accuracy: 85.71% -> 86.83% | Helped |
| Set max_words to 400 instead of 600 (ablation) | ReadAgent-P (1-5 pages) · QuALITY dev | accuracy: 86.83% -> 86.82% | No clear effect |
| Set max_words to 800 instead of 600 (ablation) | ReadAgent-P (1-5 pages) · QuALITY dev | accuracy: 86.83% -> 86.34% | Hurt |
| Set max_words to 1200 instead of 600 (ablation) | ReadAgent-P (1-5 pages) · QuALITY dev | accuracy: 86.83% -> 85.67% | Hurt |
| Include the previous page in the pagination prompt (ablation) | ReadAgent · QMSum | performance | Helped |
| Include the previous page in the pagination prompt (ablation) | ReadAgent · QuALITY | performance | No clear effect |
| Add 'Take a deep breath and tell me' to the lookup prompt (ablation) | ReadAgent · QuALITY | response quality | Helped |
| Use alternative wording in the lookup prompt (ablation) | ReadAgent · Reading comprehension tasks | performance | No clear effect |
Surprising findings
- ReadAgent with look-ups outperforms feeding the full original QuALITY article to the LLM, even though the full text is the nominal upper bound.
- Gist memory alone is a strong representation, but the paper notes the LLM may hallucinate details elided by gisting, a risk they did not study.
Limitations
- Cannot provide infinite context; performance degrades when the gist memory itself is very long.
- Gisting discards details, risking hallucination when tasks need fine-grained information.
- Retrieval is limited to per-document databases, and the gist memory size bounds how many documents can be handled.
Experimental setup
Datasets: QuALITY NarrativeQA QMSum Mind2Web
Benchmarks: QuALITY NarrativeQA QMSum Mind2Web
Metrics: accuracy LLM-Rating-1 LLM-Rating-2 ROUGE-1 ROUGE-2 ROUGE-L compression rate element accuracy operation F1 step success rate
Baselines: BM25 retrieval neural retrieval with Gemini embeddings full raw content truncated raw content GistMem MindAct GPT-3.5 Turbo
Builds on: retrieval-augmented generation MemWalker PEARL
Closest papers by meaning
- CAM: A Constructivist View of Agentic Memory for LLM-Based Reading Comprehension (NeurIPS 2025 · similarity 0.89)
- Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions (ICLR 2026 · similarity 0.89)
- Chain of Agents: Large Language Models Collaborating on Long-Context Tasks (NeurIPS 2024 · similarity 0.88)
- A-Mem: Agentic Memory for LLM Agents (NeurIPS 2025 · similarity 0.87)
- Harness the Memory: A Holistic Evaluation of Memory Substrates in Memory Agents (NeurIPS 2026 · similarity 0.87)
- AMA-Bench: Evaluating Long-Horizon Memory for Agentic Applications (ICML 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.