Research map / Large language models / Alignment and preferences

Simulating Human Memory with Language Models

NeurIPS 2026 · Large language models › Alignment and preferences · approach: LLM-as-judge and feedback data · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Compares human and LLM performance on ten classic memory tasks, finds LLMs have near-ceiling memory, and shows a key-value compactor with four slots yields more human-like score distributions and aids a teaching-document reranking proof of concept.

Problem

Language models used as user simulators have far more reliable memory than real humans, so their simulations misrepresent human cognitive limits; the paper asks how to make LLM memory human-like.

Approach

Builds a ten-task human-vs-LLM memory benchmark scored by Wasserstein-based humanlikeness and introduces Compactor, an agent that encodes input into a capacity-limited four-slot key-value memory via tool calls and answers only from that memory.

Key results

• Default prompting (TaskPr) yields near-ceiling LLM memory across all ten tasks; for digit span, models recall sequences up to length 20 versus a human average of 6.88.
• Compactor raises digit span humanlikeness by up to +0.61 for Qwen3-8B Standard relative to TaskPr, while human-simulation prompts give near-zero gains.
• In the education reranking proof of concept, Llama 3 8B Instruct with Compactor achieves a +0.180 absolute gain in pairwise reranking accuracy over TaskPr.

Surprising findings

• Explicit human-simulation prompts often leave humanlikeness unchanged or reduce it, and frontier models such as GPT-5.4 do not simulate human memory better than smaller models.
• Same-task few-shot human examples help but out-of-domain examples from another working memory task give no comparable gain.

Limitations

• Benchmark covers only text-in, text-out memory types; semantic and visual memory are not tested.
• Compactor matches humans' score distributions but produces forgetting patterns that differ from human error patterns.
• Reranking gains are inconsistent across models and the best model is below 70% pairwise accuracy.

Experimental setup

Datasets: Naturalistic Free Recall dataset

Benchmarks: Human Memory Simulation Benchmark digit span reverse digit span N-back variable mapping word recognition factual QA narrative QA narrative free recall map task craft task

Metrics: humanlikeness (1 - Wasserstein-1 distance) pairwise reranking accuracy BLEU embedding similarity

Baselines: TaskPr HumPr MemPr unstructured summarizer agent

Builds on: Centaur Cowan working memory capacity (4 chunks) naturalistic free recall paradigm

Closest papers by meaning

• Turning large language models into cognitive models (ICLR 2024 · similarity 0.89)
• CogBench: a large language model walks into a psychology lab (ICML 2024 · similarity 0.88)
• Using Artificial Populations to Study Psychological Phenomena in Neural Models (AAAI 2024 · similarity 0.87)
• Measuring Non-Adversarial Reproduction of Training Data in Large Language Models (ICLR 2025 · similarity 0.87)
• Episodic Memories Generation and Evaluation Benchmark for Large Language Models (ICLR 2025 · similarity 0.87)
• OdysSim: Building Foundation Models for Human Behavior Simulation (NeurIPS 2026 · similarity 0.87)