Research map / Large language models / Reasoning and chain of thought

Latent Planning Emerges with Scale

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Search and self-correction in reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Defines causal criteria for latent planning in LLMs and uses transcoder feature circuits on Qwen-3 (0.6B-14B) to show planning on a/an agreement, rhyming couplets, and steering-based prose planning, with forward planning improving faster than backward planning as scale grows.

Problem

Whether LLMs plan implicitly (latent planning) rather than only observationally decodable future tokens; prior evidence is largely correlational, and causal evidence has been limited to closed models.

Approach

Define latent planning via two causal conditions (forward planning: a representation causes generation of a future token; backward planning: it causes preceding context that licenses that token), and test them with transcoder feature circuits, interventions, and steering across Qwen-3 sizes.

Key results

• Only Qwen-3 14B reliably recalls the minority-class article 'an' (>0.8 recall); 0.6-1.7B models always predict 'a'.
• Ablating planning features harms and upweighting them improves correct-article probability mainly on 'an' examples; direct-effect-only interventions are much weaker.
• On rhyming couplets, Qwen-3 14B reaches about 0.6 rhyme accuracy (0.8 assonant), but steering on rhyme features yields little evidence of backward planning.

Surprising findings

• Qwen-3 4B and 8B have planning features on a/an despite failing the task, suggesting nascent planning.
• Steering on rhyme features changes the final rhyme but the steered intermediate context does not license the new rhyme better, indicating weak backward planning.

Limitations

• Only one model family (Qwen-3) studied, limiting cross-family comparison.
• Simple tasks only; models up to 14B struggle with complex planning.
• Rhyming feature identification via heuristics with moderate recall; backward planning evidence mostly negative.

Experimental setup

Datasets: TinyStories

Benchmarks: a/an agreement dataset is/are agreement dataset el/la agreement dataset rhyming couplet dataset (985 first lines) TinyStories steering set (100 inputs)

Metrics: per-class recall rhyme accuracy proportion of influence through planning nodes change in p(correct article) macro F1 coherence and word-containment rates

Baselines: random feature interventions Llama-3-8B-Instruct probing (Dong et al.-style animal probing)

Builds on: transcoders attribution graphs / circuit-tracer Lindsey et al. poetry planning study Patchscopes

Closest papers by meaning

• Emergent Response Planning in LLMs (ICML 2025 · similarity 0.90)
• How Far Ahead Do LLMs Plan? Uncovering the Latent Horizon in Chain-of-Thought Reasoning (ICML 2026 · similarity 0.87)
• ALPINE: Unveiling The Planning Capability of Autoregressive Learning in Language Models (NeurIPS 2024 · similarity 0.86)
• LLMs Can Plan Only If We Tell Them (ICLR 2025 · similarity 0.86)
• Large Language Models as Planning Domain Generators (Student Abstract) (AAAI 2024 · similarity 0.86)
• On the self-verification limitations of large language models on reasoning and planning tasks (ICLR 2025 · similarity 0.85)