Research map / Large language models / Retrieval and knowledge
From Isolated Conversations to Hierarchical Schemas: Dynamic Tree Memory Representation for LLMs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MemTree, an online tree-structured memory in which new information is inserted top-down via embedding similarity with LLM-based summary aggregation along the path, evaluated against flat and offline hierarchical memory methods on dialogue and document QA.
Problem
LLM long-term memory is typically a flat lookup table of isolated entries, which fails to capture structure and cannot be updated online by offline hierarchical methods (RAPTOR, GraphRAG) without full rebuilds.
Approach
Maintain a dynamic tree memory where new content is routed top-down by cosine similarity against an adaptive depth-dependent threshold, creating leaves or merging into parents; ancestor summaries are updated by LLM aggregation, enabling online hierarchical memory with collapsed-tree retrieval.
Key results
- On MSC-E (200 rounds), MemTree reaches 82.5% overall accuracy versus 80.7% for MemoryStream and 78.0% for full history with GPT-4o.
- On MultiHop RAG, MemTree achieves 80.5% overall accuracy, above MemoryStream (74.7%) and within 0.5 points of offline RAPTOR (81.0%), and best on temporal queries (68.4%).
- On QuALITY, MemTree gets 59.8% overall accuracy versus 43.8% for MemoryStream, close to RAPTOR (59.0%), while full-text Llama-3.1-70B reaches 65.1%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace flat MemoryStream memory with dynamic tree memory | GPT-4o on MSC-E · MSC-E | overall accuracy: 80.7 -> 82.5 | Helped |
| replace flat MemoryStream memory with dynamic tree memory | GPT-4o on MSC · MSC | accuracy: 84.4 -> 84.8 | No clear effect |
| replace flat MemoryStream memory with dynamic tree memory | Llama-3.1-70B on QuALITY · QuALITY Overall | accuracy: 43.8 -> 59.8 | Helped |
| replace flat MemoryStream memory with dynamic tree memory | GPT-4o on MultiHop RAG · MultiHop RAG Comparison | accuracy: 64.8 -> 73.9 | Helped |
| replace flat MemoryStream memory with dynamic tree memory | GPT-4o on MultiHop RAG · MultiHop RAG Temporal | accuracy: 59.3 -> 68.4 | Helped |
| use traversal retrieval instead of collapsed retrieval with k ≤ 5 (ablation) | MemTree · MultiHop RAG, especially temporal queries | accuracy | Hurt |
| use traversal retrieval with k=10 instead of collapsed retrieval (ablation) | MemTree · MultiHop RAG | accuracy | No clear effect |
| construct the tree with GPT-4o-mini and text-embed-3-small instead of GPT-4o and text-embed-3-large (ablation) | MemTree · MultiHop RAG Overall | accuracy: 80.5 -> 78.4 | Hurt |
| construct the tree with Llama-3.1-8B and text-embed-3-small instead of GPT-4o and text-embed-3-large (ablation) | MemTree · MultiHop RAG Overall | accuracy: 80.5 -> 78.1 | Hurt |
| vary the adaptive threshold parameters θ0 and λ (ablation) | MemTree · MultiHop RAG Overall | accuracy | No clear effect |
Surprising findings
- On short MSC dialogues, feeding the full history to GPT-4o (95.6%) beats all external memory methods.
- Concatenating a summary of the full history drops accuracy to 35% on MSC, far below memory-based retrieval.
Limitations
- Cumulative construction cost is about 1.4x higher than offline RAPTOR/GraphRAG.
- Sequential insertion biases the memory toward recent conversation content.
- Traversal retrieval with small top-k loses accuracy, so collapsed retrieval is needed; MemGPT results could not be reproduced.
Experimental setup
Datasets: Multi-Session Chat MSC-E (extended, generated with GPT-4o-style prompting) QuALITY MultiHop RAG
Benchmarks: Multi-Session Chat MSC-E QuALITY MultiHop RAG
Metrics: accuracy ROUGE-L recall
Baselines: MemoryStream MemGPT RAPTOR GraphRAG Full history (naive concatenation)
Builds on: RAPTOR MemoryStream MemGPT GraphRAG OTD online top-down clustering
Closest papers by meaning
- StateTree: Enhancing Long-term Dialogue Reasoning via Reinforcement Learning (NeurIPS 2026 · similarity 0.89)
- LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory (ICLR 2025 · similarity 0.88)
- AdmTree: Compressing Lengthy Context with Adaptive Semantic Trees (NeurIPS 2025 · similarity 0.88)
- Hierarchical Context Merging: Better Long Context Understanding for Pre-trained LLMs (ICLR 2024 · similarity 0.88)
- PlugMem: A Task-Agnostic Plugin Memory Module for LLM Agents (ICML 2026 · similarity 0.87)
- A-Mem: Agentic Memory for LLM Agents (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.