Research map / Large language models / Reasoning and chain of thought
Enhancing Chain of Thought Prompting in Large Language Models via Reasoning Patterns
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Pattern-CoT, which selects diverse CoT demonstrations by clustering rationales according to task-specific reasoning-operation patterns instead of question semantics, improving unsupervised CoT on eight reasoning benchmarks.
Problem
Unsupervised CoT demonstration selection relies on question semantics, which introduces noise and offers little interpretability about why demonstrations help reasoning.
Approach
Extract task-specific reasoning operation tokens (e.g., +, -, x, /, 'twice', 'tails up') from zero-shot rationales to form pattern sequences, encode them with a sentence encoder, cluster them, and sample one demonstration per cluster; also proposes a formula for choosing the number of clusters k from the number of operation types and dataset size.
Key results
- On LLaMA-2-13b-chat, Pattern-CoT with adaptive k reaches 38.44% on GSM8K and 60.97% on Date vs. 36.77% and 53.93% for Auto-CoT.
- On LLaMA-2-7b-chat, Pattern-CoT improves SingleEq to 71.85% and SVAMP to 48.50% over Auto-CoT's 64.96% and 43.80%.
- On Qwen-7B-chat, Pattern-CoT improves AddSub from 62.53% (Auto-CoT) to 67.59%; on GPT-3.5-turbo it reaches 93.11% on SingleEq vs. 91.53% for Auto-CoT.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Select demonstrations using reasoning patterns instead of semantic-based selection | Pattern-CoT on LLaMA-2-7B-chat-hf · GSM8K | accuracy (%): 21.00 -> 27.45 | Helped |
| Select demonstrations using reasoning patterns instead of semantic-based selection | Pattern-CoT on LLaMA-2-7B-chat-hf · AQuA | accuracy (%): 24.01 -> 28.34 | Helped |
| Select demonstrations using reasoning patterns instead of semantic-based selection | Pattern-CoT on LLaMA-2-13B-chat-hf · Coin | accuracy (%): 62.80 -> 57.40 | Mixed |
| Use adaptive k (ablation) | Pattern-CoT on LLaMA-2-7B-chat-hf · GSM8K | accuracy (%): 27.45 -> 28.05 | Helped |
| Use adaptive k (ablation) | Pattern-CoT on LLaMA-2-7B-chat-hf · AddSub | accuracy (%): 65.06 -> 67.08 | Helped |
| Use adaptive k (ablation) | Pattern-CoT on LLaMA-2-7B-chat-hf · AQuA | accuracy (%): 28.34 -> 29.13 | Helped |
| Use adaptive k (ablation) | Pattern-CoT on LLaMA-2-7B-chat-hf · Coin | accuracy (%): 59.40 -> 58.40 | No clear effect |
| Use adaptive k (ablation) | Pattern-CoT on LLaMA-2-13B-chat-hf · GSM8K | accuracy (%): 37.68 -> 38.44 | Helped |
| Use adaptive k (ablation) | Pattern-CoT on LLaMA-2-13B-chat-hf · AddSub | accuracy (%): 65.82 -> 64.81 | Hurt |
| Use adaptive k (ablation) | Pattern-CoT on LLaMA-2-13B-chat-hf · AQuA | accuracy (%): 26.37 -> 31.49 | Helped |
| Use adaptive k (ablation) | Pattern-CoT on LLaMA-2-13B-chat-hf · Coin | accuracy (%): 57.40 -> 67.80 | Helped |
| Use only a subset of the operation tokens rather than the full set (ablation) | Pattern-CoT on LLaMA-2-7B-chat-hf · GSM8K and AQuA | accuracy (%) | Hurt |
Surprising findings
- Most selected demonstrations contain incorrect answers (e.g., 100% error rate on AQuA), yet the method still improves performance, suggesting LLMs learn reasoning patterns rather than correctness.
- Auto-CoT sometimes underperforms direct answering on GPT-3.5 and Qwen, which the authors attribute to semantic noise.
Limitations
- Operation token sets are hand-crafted or GPT-4/LLM-generated per task, limiting scalability to tasks without clear operations.
- Evaluation limited to 7B/13B LLaMA-2 models plus one GPT-3.5 and one Qwen-7B model due to hardware and budget constraints.
- Improvements on GSM8K and AQuA are modest and the adaptive k formula is described as empirical.
Experimental setup
Datasets: MultiArith GSM8K AddSub AQuA SingleEq SVAMP Coin-Flip BIG-bench Date Understanding
Benchmarks: MultiArith GSM8K AddSub AQuA SingleEq SVAMP Coin-Flip BIG-bench Date Understanding
Metrics: accuracy
Baselines: Zero-Shot-CoT Random-CoT Auto-CoT Auto-CoT-RA Self-Consistency
Builds on: Auto-CoT Zero-Shot-CoT Self-Consistency
Closest papers by meaning
- Large Language Models as Analogical Reasoners (ICLR 2024 · similarity 0.93)
- SCOUT: Teaching Pre-trained Language Models to Enhance Reasoning via Flow Chain-of-Thought (NeurIPS 2025 · similarity 0.93)
- Instance-adaptive Zero-shot Chain-of-Thought Prompting (NeurIPS 2024 · similarity 0.93)
- Chain-of-Thought Reasoning Without Prompting (NeurIPS 2024 · similarity 0.92)
- How Does Chain of Thought Think? Mechanistic Interpretability of Chain-of-Thought Reasoning with Sparse Autoencoding (AAAI 2026 · similarity 0.92)
- CoT Vectors: Transferring and Probing the Reasoning Mechanisms of LLMs (ICLR 2026 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.