Research map / Large language models / Language, safety and interpretability
Internal Planning in Language Models: Characterizing Horizon and Branch Awareness
Muhammed Ustaomeroglu, Baris Askin, Gauri Joshi, Carlee Joe-Wong, Guannan Qu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train with MTP loss instead of NTP loss | GPT-3 Small · PF-Long | accuracy: 0.60 -> 0.85 | Improved |
| train with MTP loss instead of NTP loss | GPT-3 Small · PF-Long | alt/decoy MI ratio: 1.45 -> 1.82 | Improved |
| train with MTP loss instead of NTP loss | GPT-3 Small · PF-Short | accuracy: 0.92 -> 0.88 | No clear effect |
| train with MTP loss instead of NTP loss | GPT-3 Small · PF-Short | alt/decoy MI ratio: 7.60 -> 6.29 | Hurt |
| train with MTP loss instead of NTP loss | GPT-3 Small · CFG | nMI decay between prefix and decision state | No clear effect |
| increase VQ-VAE codebook size K (ablation) | VQ-VAE MI estimator · Synthetic known-MI validation | agreement of normalized MI with ground truth | Improved |
| use a task requiring longer paths (PF instead of CFG) | GPT-3 Small · CFG vs path finding | nMI between prefix and later decision states: nMI at tau=10 about one-fifth of initial on CFG | Improved |
Closest papers by meaning
- ALPINE: Unveiling The Planning Capability of Autoregressive Learning in Language Models (NeurIPS 2024 · similarity 0.91)
- Non-myopic Generation of Language Models for Reasoning and Planning (ICLR 2025 · similarity 0.89)
- How Far Ahead Do LLMs Plan? Uncovering the Latent Horizon in Chain-of-Thought Reasoning (ICML 2026 · similarity 0.89)
- Emergent Response Planning in LLMs (ICML 2025 · similarity 0.89)
- LLMs Can Plan Only If We Tell Them (ICLR 2025 · similarity 0.89)
- Large Language Models as Planning Domain Generators (Student Abstract) (AAAI 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
