Research map / Multi-agent systems and games / LLM agents

Memory-T1: Reinforcement Learning for Temporal Reasoning in Multi-session Agents

ICLR 2026 · Multi-agent systems and games › LLM agents · approach: Self-evolving language agent learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Memory-T1 couples a temporal-filter plus BM25 candidate pipeline with a GRPO-trained selector that is rewarded for answer accuracy, session-evidence F1, and session- and utterance-level temporal consistency, reaching 67.0% on Time-Dialog with a 7B model.

Problem

Temporal reasoning over long, noisy multi-session dialogue histories, where long-context LLMs struggle to identify temporally relevant evidence and answer-only RL rewards are too sparse.

Approach

A coarse-to-fine memory pipeline (LLM-predicted temporal window filter, then BM25 relevance) followed by an RL-trained evidence-selection policy whose dense reward adds session-level chronological proximity (logistic penalty on day gap) and utterance-level event-time fidelity to standard accuracy and evidence F1 rewards.

Key results

• Memory-T1 (7B) reaches 67.0% overall on Time-Dialog, versus 60.7% for Qwen2.5-14B and 53.2% for Qwen2.5-7B-Instruct.
• Memory-T1 (3B) reaches 66.9% overall on Time-Dialog, outperforming Qwen2.5-14B despite being much smaller.
• On LoCoMo (out-of-domain), Memory-T1 (3B) Non-RAG reaches 37.7% overall versus 33.5% for Qwen-2.5-3B-Instruct.

Surprising findings

• Ablating only the session-level chronological proximity reward improves Category A tasks by 23.4% while collapsing Category B reasoning by 56.2%.
• Memory-T1 performs better in the Non-RAG setting than in the RAG setting on LoCoMo, except for the adversarial subset.

Limitations

• Near-zero scores on Comparison and Timeline tasks indicate limits on compositional temporal logic.
• Relies on GPT-4-assisted annotations (about 95% session-level accuracy) for temporal ranges and evidence, verified by humans.
• Reward weights and temporal hyperparameters were chosen heuristically with limited sensitivity analysis.

Experimental setup

Datasets: Time-Dialog LoCoMo

Benchmarks: Time-Dialog LoCoMo

Metrics: accuracy F1 recall

Baselines: Full Context RAG ReAct (GPT-4) SFT MemAgent Time-R1 RL (task reward only)

Builds on: GRPO Time-R1 TReMu Time dataset

Closest papers by meaning

• StateTree: Enhancing Long-term Dialogue Reasoning via Reinforcement Learning (NeurIPS 2026 · similarity 0.91)
• MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agents (ICLR 2026 · similarity 0.91)
• Remember the Decision, Not the Description: A Rate-Distortion Framework for Agent Memory (NeurIPS 2026 · similarity 0.90)
• REMem: Reasoning with Episodic Memory in Language Agent (ICLR 2026 · similarity 0.90)
• Rethinking How to Remember: Beyond Atomic Facts in Lifelong LLM Agent Memory (NeurIPS 2026 · similarity 0.90)
• Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions (ICLR 2026 · similarity 0.90)