Research map / Large language models / Reasoning and chain of thought

StateTree: Enhancing Long-term Dialogue Reasoning via Reinforcement Learning

NeurIPS 2026 · Large language models › Reasoning and chain of thought · approach: Test-time compute scaling · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes StateTree, a curriculum GRPO pseudo-task that embeds tree-structured key-value path-tracing records into 616 LoCoMo dialogues, improving long-term dialogue reasoning and generalizing from 10K to 128K contexts.

Problem

Long-term multi-session dialogue reasoning requires cross-session retrieval, temporal reasoning, knowledge update and multi-hop composition, but training data is scarce and full-length long-context RL is computationally prohibitive.

Approach

Augment scarce multi-session dialogues with binary-tree key-value records scattered across sessions (same key with different timestamps at each fork) so the model must root-to-leaf trace by comparing timestamps to recover a hidden target question; add a compositional variant whose edges carry step-level fragments, trained via a four-stage curriculum with GRPO and a max(EM, LLM-judge) reward.

Key results

• StateTree-14B reaches 60.91% average accuracy on LoCoMo vs 49.09% for Qwen2.5-14B-Instruct and 46.36% for QwenLong-L1-32B.
• StateTree-14B achieves 59.00% on LongMemEval (128k) vs QwenLong-L1-32B at 45.20%, despite training only at 10K tokens.
• StateTree-8B preserves short-context reasoning (avg 80.1 vs 80.0 for Qwen3-8B on MMLU/MATH/IFEval).

Surprising findings

• Generic chain-of-thought prompting yields only marginal gains (45.19 vs 44.29 on LoCoMo), suggesting structural navigation rather than reasoning depth is the bottleneck.
• Using random UUID keys versus word keys matters: replacing UUIDs with common words retains about 80% of the gain, implying semantic keys invite shortcuts.

Limitations

• All training data derives from a single English dataset (LoCoMo), so cross-language and cross-domain generalization is untested.
• The Compositional StateTree depends on GPT-4o for question decomposition and distractor generation, with quality bounded by the LLM.
• Tree depths are fixed (D=2 and D=3); adaptive or dynamic tree structures are not explored.

Experimental setup

Datasets: LoCoMo DAPO-Math

Benchmarks: LoCoMo LongMemEval PersonaMem MMLU MATH-500 IFEval

Metrics: accuracy F1 BLEU-1 exact match

Baselines: GPT-4o QwenLong-L1-32B R1-Distill-Qwen-32B SEALONG LoongRL RL-MemAgent

Builds on: GRPO LoCoMo LoongRL SEALONG DAPO

Closest papers by meaning

• Memory-T1: Reinforcement Learning for Temporal Reasoning in Multi-session Agents (ICLR 2026 · similarity 0.91)
• From Isolated Conversations to Hierarchical Schemas: Dynamic Tree Memory Representation for LLMs (ICLR 2025 · similarity 0.89)
• LoongRL: Reinforcement Learning for Advanced Reasoning over Long Contexts (ICLR 2026 · similarity 0.88)
• TA&AT: Enhancing Task-Oriented Dialog with Turn-Level Auxiliary Tasks and Action-Tree Based Scheduled Sampling (AAAI 2024 · similarity 0.88)
• Rethinking How to Remember: Beyond Atomic Facts in Lifelong LLM Agent Memory (NeurIPS 2026 · similarity 0.88)
• Learning to Clarify: Multi-turn Conversations with Action-Based Contrastive Self-Training (ICLR 2025 · similarity 0.87)