Research map / Large language models / Reasoning and chain of thought

Reprompting: Automated Chain-of-Thought Prompt Inference Through Gibbs Sampling

ICML 2024 · Large language models › Reasoning and chain of thought · approach: Theory of autoregressive reasoning · 4 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces Reprompting, a Gibbs-sampling-based iterative algorithm that automatically infers chain-of-thought recipes for each LLM from a few labeled training problems, without human-written CoT.

Problem

Hand-crafting chain-of-thought prompts requires costly human expertise, and CoT prompts that work well for one LLM may be suboptimal for another, making fair LLM comparisons difficult.

Approach

Frames CoT prompt discovery as sampling from a joint distribution of recipes over training problems, using Gibbs sampling: new recipes are generated by prompting the LLM with K previously sampled recipes as parents, and rejection sampling keeps recipes that lead to correct answers. An evolutionary-like recombination and selection process converges to a set of effective recipes.

Key results

• Reprompting with ChatGPT averages 83.0 on five BBH tasks vs 70.1 for CoT and 72.2 for Auto-CoT, and over 20 tasks it is reported as +9.4 points over human-written CoT on average.
• On seven additional tasks, Reprompting averages 53.0 vs 44.9 for CoT and 38.3 for zero-shot with ChatGPT.
• Using ChatGPT for initialization and InstructGPT for sampling lifts InstructGPT Temporal Sequences accuracy to 99.2 vs 28.4 for InstructGPT alone in the main table.

Measured effects reported in the paper

Surprising findings

• CoT recipes optimized for one LLM can perform 18-19 points worse on another LLM, so LLM comparisons depend heavily on the CoT prompt used.
• Even erroneous model-generated recipes can evolve into high-quality recipes through iterative recombination.

Limitations

• Sensitive to the initialization model; InstructGPT alone sometimes fails to discover good recipes without ChatGPT initialization.
• Recipes optimized for one LLM may transfer poorly to others, requiring per-model optimization cost.
• Relies on labeled training answers for rejection sampling, so it is not fully unsupervised.

Experimental setup

Datasets: Big-Bench Hard GSM8K MATH

Benchmarks: Big-Bench Hard GSM8K MATH

Metrics: exact-match accuracy

Baselines: zero-shot prompting few-shot prompting CoT CoT+self-consistency Auto-CoT APO

Builds on: chain-of-thought prompting self-consistency Auto-CoT automatic prompt optimization

Closest papers by meaning

• Training Nonlinear Transformers for Chain-of-Thought Inference: A Theoretical Generalization Analysis (ICLR 2025 · similarity 0.86)
• SCOUT: Teaching Pre-trained Language Models to Enhance Reasoning via Flow Chain-of-Thought (NeurIPS 2025 · similarity 0.86)
• Enhancing Chain of Thought Prompting in Large Language Models via Reasoning Patterns (AAAI 2025 · similarity 0.86)
• Optimizing Chain-of-Thought Reasoners via Gradient Variance Minimization in Rejection Sampling and RL (NeurIPS 2025 · similarity 0.86)
• How Does Chain of Thought Think? Mechanistic Interpretability of Chain-of-Thought Reasoning with Sparse Autoencoding (AAAI 2026 · similarity 0.85)
• Continuous Chain of Thought Enables Parallel Exploration and Reasoning (ICLR 2026 · similarity 0.85)