Research map / Continual learning / Continual fine-tuning of LLMs

Tell Me What To Learn: Generalizing Neural Memory to be Controllable in Natural Language

NeurIPS 2026 · Continual learning › Continual fine-tuning of LLMs · approach: Continual pretraining of language models · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes generalized neural memory (GNM), a MemoryLLM-based system whose memory updates are conditioned on natural-language learning instructions, and evaluates it on a synthetic CounterFact-derived benchmark against ICL-FT and RAG-FT baselines.

Problem

Existing neural memory systems optimize a single fixed update objective and give users no way to specify, in natural language, what should be learned or ignored from heterogeneous document streams.

Approach

Make the memory write step instruction-conditioned: the update rule takes the current memory, a natural-language learning instruction, and a document, and is trained end-to-end (with gradients through the memory write) so that the memory selectively stores instructed content and ignores the rest, generalizing to unseen instructions.

Key results

• GNM achieves close to ICL-FT accuracy on held-out instructions while outperforming all baselines on overall harmonic-mean score, with substantial gains in fact selectivity.
• On compositional instructions, GNM achieves about twice RAG-FT's fact selectivity and over ten times ICL-FT's.
• Format accuracy on unseen formats: RAG-FT and ICL-FT fail to generalize while GNM generalizes well; GNM scales O(1) in cost with documents seen.

Surprising findings

• Ablating gradients through the memory write step eliminates the selectivity and format gains, indicating learning what to write is critical.
• Even when fine-tuned, ICL-FT and RAG-FT struggle to ignore information provided in context.

Limitations

• Evaluated only on a synthetic benchmark; no real-world data.
• MemoryLLM overwrites memory, so retention degrades exponentially over about 20 time steps.
• Performance is poor on inconsistent information across documents, likely due to lack of update-order mechanism.

Experimental setup

Datasets: CounterFact synthetic instruction-document benchmark built from CounterFact

Benchmarks: synthetic language-controlled memory benchmark (test-ood)

Metrics: fact accuracy fact specificity fact selectivity format accuracy refusal precision refusal recall harmonic mean score FLOPs per token

Baselines: MemoryLLM ICL-FT RAG-FT GNM (Ablation)

Builds on: MemoryLLM CounterFact ROME

Closest papers by meaning

• MemGen: Weaving Generative Latent Memory for Self-Evolving Agents (ICLR 2026 · similarity 0.86)
• Remember the Decision, Not the Description: A Rate-Distortion Framework for Agent Memory (NeurIPS 2026 · similarity 0.86)
• AdaMEM: Test-Time Adaptive Memory for Language Agents (ICML 2026 · similarity 0.86)
• A-Mem: Agentic Memory for LLM Agents (NeurIPS 2025 · similarity 0.86)
• MemSkill: Learning and Evolving Memory Skills for Self-Evolving Agents (NeurIPS 2026 · similarity 0.85)
• Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions (ICLR 2026 · similarity 0.85)