Research map / Large language models / Reasoning and chain of thought

TSLM: Tree-Structured Language Modeling for Divergent Thinking

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Test-time compute scaling · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Tree-Structured Language Modeling (TSLM), which serializes search trees with special tokens ([SEP], [FAIL], [GOAL]) and trains models via SFT to generate branching exploration within a single generation.

Problem

Sequential language models commit to single reasoning paths and cannot explicitly explore and decouple multiple branches, while external search methods like Tree-of-Thought require many independent model calls.

Approach

Encode complete search trees (including failed branches) as linear token sequences with structural markers, and train with standard LM loss so the model internalizes systematic branching; at inference, parse branches and expand them selectively, with a claimed context-decoupling effect on sibling/ancestor conditioning.

Key results

• TSLM reaches 100% pass@1 on Game of 24 vs 17% for SC and 15% for GRPO and 32% for ToT@100.
• On 20x20 Gridworld (o.o.d), TSLM gets 91.5% vs 42.7% for ToT@100 and 81.1% for PC.
• On GSM8K, TSLM pass@1 is 61.6% vs ToT@100 62.3%; scaling candidates reaches 67.2% convergence.

Surprising findings

• TSLM correctly rejected 97 of 100 unsolvable Game of 24 instances, while SC, PC and GRPO identified none.
• Baseline ToT is evaluated at pass@100 while TSLM is pass@1, a comparison setup the paper itself frames as conservative for TSLM.

Limitations

• Training on full trees increases computational cost roughly by average tree size.
• Synthetic tree quality depends heavily on the supervision model; Llama-3-8B yields poor trees for GSM8K.
• Requires verifiable solutions to construct training trees, limiting open-ended domains.

Experimental setup

Datasets: Game of 24 Textualized Gridworld ProntoQA GSM8K

Benchmarks: Game of 24 Textualized Gridworld 10x10 Textualized Gridworld 20x20 (o.o.d) ProntoQA GSM8K

Metrics: pass@1 pass@100 success rate wall-clock inference time

Baselines: Sequence Cloning (SC) Procedure Cloning (PC) GRPO Tree-of-Thought (ToT)

Builds on: Tree-of-Thoughts RAP GRPO Chain-of-Thought

Closest papers by meaning

• AlphaZero-Like Tree-Search can Guide Large Language Model Decoding and Training (ICML 2024 · similarity 0.90)
• Algorithm of Thoughts: Enhancing Exploration of Ideas in Large Language Models (ICML 2024 · similarity 0.89)
• T1: Advancing Language Model Reasoning through Reinforcement Learning and Inference Scaling (ICML 2025 · similarity 0.89)
• Policy Guided Tree Search for Enhanced LLM Reasoning (ICML 2025 · similarity 0.89)
• Language Agent Tree Search Unifies Reasoning, Acting, and Planning in Language Models (ICML 2024 · similarity 0.88)
• Soft Reasoning: Navigating Solution Spaces in Large Language Models through Controlled Embedding Exploration (ICML 2025 · similarity 0.88)