Research map / Large language models / Reasoning and chain of thought
Why think step by step? Reasoning emerges from the locality of experience
Ben Prystawski, Michael Y. Li, Noah Goodman
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| generate intermediate variables (free/scaffolded) before target instead of direct prediction (ablation) | autoregressive transformer on Bayes net samples · held-out high-MI pairs, local training | MSE to true conditional probability | Improved |
| generate intermediate variables when training data is fully observed (ablation) | autoregressive transformer · held-out pairs, fully observed training | MSE to true conditional probability | No clear effect |
| use wrong-locality training data (ablation) | autoregressive transformer · held-out pairs | MSE to true conditional probability: MSE vs true on seen pairs .004 -> .052 | Hurt |
| use irrelevant (negative) scaffold variables instead of relevant scaffold (ablation) | scaffolded generation · held-out pairs, local training | MSE to true conditional probability | Hurt |
| combine local training data with free generation reasoning | autoregressive transformer · data efficiency | training tokens to match conditional probabilities: direct prediction on fully observed data needs over 3x more training | Improved |
| resample intermediate variables more times (more Monte Carlo samples) (ablation) | free/scaffolded generation · held-out pairs | MSE | Improved |
| train on local data, direct prediction on pairs that co-occur (ablation) | direct prediction · non-held-out pairs | MSE to true conditional probability: .013 (fully observed) -> .004 (local geom) | Improved |
Closest papers by meaning
- SCOUT: Teaching Pre-trained Language Models to Enhance Reasoning via Flow Chain-of-Thought (NeurIPS 2025 · similarity 0.89)
- Causal Sufficiency and Necessity Improves Chain-of-Thought Reasoning (NeurIPS 2025 · similarity 0.89)
- Training Language Models to Reason Efficiently (NeurIPS 2025 · similarity 0.88)
- Understanding Reasoning Ability of Language Models From the Perspective of Reasoning Paths Aggregation (ICML 2024 · similarity 0.88)
- Exploring the Limits of Sub-Billion Language Model Reasoners with Open Training Recipes (ICLR 2026 · similarity 0.88)
- Long Chain-of-Thought Reasoning Across Languages (ICLR 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
