Research map / Large language models / Reasoning and chain of thought
Understanding Reasoning Ability of Language Models From the Perspective of Reasoning Paths Aggregation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes viewing LM reasoning as aggregating random-walk reasoning paths from pre-training data, validated on KG reasoning with GPT-2 trained from scratch and on CoT tasks via random-walk continued pre-training of LoRA-tuned LLMs.
Problem
How next-token-prediction pre-training gives rise to LM reasoning ability, both over knowledge graphs and in chain-of-thought math/logic reasoning; and whether unlabeled reasoning paths can improve multi-step reasoning.
Approach
Models LM reasoning as a weighted aggregation of random-walk path probabilities (PRA-inspired) over KGs, and proposes building random-walk reasoning paths from CoT steps via a latent reasoning graph (k-means clustering of reasoning states) to continue pre-train LLMs before SFT.
Key results
- On CoT tasks, random-walk continued pre-training plus SFT improves average accuracy over SFT across four models, e.g., Llama 2 7B from 44.8 to 47.7 and Llama 2 13B from 52.7 to 55.3.
- Random-walk augmentation gives largest gain on StrategyQA (e.g., Gemma 2B 54.2 to 56.3 and Llama 2 7B 58.4 to 63.7), attributed to simpler reasoning.
- On KG reasoning, LM distribution is close to weighted path aggregation, with KL divergence reflecting rule weighting; accuracy peaks at an intermediate random-walk path length.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add random-walk path training before SFT | Gemma (2B) · GSM8K, AQUA, SVAMP, StrategyQA, LogicalDeduction | accuracy: 24.8 -> 26.1; 31.4 -> 33.9; 56.4 -> 60.3; 54.2 -> 56.3; 50.7 -> 51.6 | Helped |
| add random-walk path training before SFT | Yi (6B) · GSM8K, AQUA, SVAMP, StrategyQA, LogicalDeduction | accuracy: 32.2 -> 33.1; 37.0 -> 39.8; 65.8 -> 67.0; 65.8 -> 70.0; 62.2 -> 63.3 | Helped |
| add random-walk path training before SFT | Llama 2 (7B) · GSM8K, AQUA, SVAMP, StrategyQA, LogicalDeduction | accuracy: 26.8 -> 28.5; 30.0 -> 34.6; 53.3 -> 55.8; 58.4 -> 63.7; 55.3 -> 56.1 | Helped |
| add random-walk path training before SFT | Llama 2 (13B) · GSM8K, AQUA, SVAMP, StrategyQA, LogicalDeduction | accuracy: 37.1 -> 41.2; 35.0 -> 37.4; 66.4 -> 69.0; 69.5 -> 71.2; 55.7 -> 57.7 | Helped |
| use a multi-step random walk instead of path length one | CoT random-walk training · GSM8K, AQUA, SVAMP | accuracy | Helped |
| increase random-walk path length beyond the optimal range | CoT random-walk training · GSM8K, AQUA, SVAMP | accuracy | Hurt |
| increase random-walk training steps from 0 to 200 (ablation) | Llama 2 (7B) · GSM8K | accuracy: 26.8 -> 27.5 | Helped |
| increase random-walk training steps from 500 to 1000 (ablation) | Llama 2 (7B) · GSM8K, AQUA, SVAMP | accuracy: 28.5 -> 24.9; 34.6 -> 32.3; 55.8 -> 51.6 | Hurt |
| use 100 clusters/nodes instead of 0 (SFT baseline) (ablation) | Llama 2 (7B) · GSM8K, AQUA, SVAMP | accuracy: 26.8 -> 28.5; 30.0 -> 34.6; 53.3 -> 55.8 | Helped |
| increase clusters/nodes from 100 to 200 (ablation) | Llama 2 (7B) · GSM8K, AQUA, SVAMP | accuracy: 28.5 -> 26.6; 34.6 -> 31.1; 55.8 -> 52.5 | Hurt |
Surprising findings
- LM trained with random walk paths generalizes to reasoning lengths beyond training path length on Countries.
- Accuracy drops for overly long random walk paths, attributed to noisy unrelated triples.
Limitations
- Experiments are small-scale and not applied to actual large-scale pre-training corpora.
- Random walk augmentation effectively up-samples the training set and may amplify artifacts or biases.
- Hyperparameters such as steps and cluster count are sensitive; large cluster counts add noise on small datasets.
Experimental setup
Datasets: Countries UMLS Kinship NELL-995 FB15K-237 GSM8K AQUA SVAMP MAWPS ASDiv-A StrategyQA LogicalDeduction (BIG-bench)
Benchmarks: Countries UMLS Kinship NELL-995 FB15K-237 GSM8K AQUA SVAMP StrategyQA LogicalDeduction
Metrics: accuracy KL divergence
Baselines: supervised fine-tuning (SFT) path ranking algorithm (PRA)
Builds on: path ranking algorithm chain-of-thought prompting LoRA GPT-2
Closest papers by meaning
- SCOUT: Teaching Pre-trained Language Models to Enhance Reasoning via Flow Chain-of-Thought (NeurIPS 2025 · similarity 0.90)
- Finding the Minimal Parameter Budget for Implicit Reasoning: A Data Complexity Driven Scaling Law for Language Models (ICML 2026 · similarity 0.89)
- MindMap: Constructing Evidence Chains for Multi-Step Reasoning in Large Language Models (AAAI 2024 · similarity 0.89)
- Long Chain-of-Thought Reasoning Across Languages (ICLR 2026 · similarity 0.89)
- Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning (ICLR 2024 · similarity 0.89)
- On the Bias of Next-Token Predictors Toward Systematically Inefficient Reasoning: A Shortest-Path Case Study (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.