Research map / Large language models / Reasoning and chain of thought

Scaling Prompt Synthesis for Large Language Model Reasoning

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Routing, pruning and distillation for reasoning

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes PromptCoT 2.0, an EM-based rationale-driven prompt synthesis framework whose synthetic math and code problems support self-play (DPO) and SFT post-training, reaching SOTA at 30B and strong 7B results.

Problem

High-quality training problems for LLM reasoning are scarce; human-curated data is costly and existing synthetic prompt corpora are too easy, narrow, and rely on hand-crafted prompt engineering.

Approach

Replaces hand-crafted rationale-prompting heuristics with an EM loop: the E-step trains a rationale generator to maximize the joint likelihood of rationale and prompt (reward = log p(x|z,c) + log p(z|c)), and the M-step trains the prompt generator on posterior-sampled concept-rationale-prompt triples, yielding harder and more diverse problems across math and code.

Key results

• Self-play on Qwen3-30B-A3B-Thinking-2507 reaches AIME 24 92.1 (from 87.7), AIME 25 89.8 (from 85.0), HMMT Feb 25 76.7 (from 71.4), LCB v5 74.2, LCB v6 71.0, Codeforces 2079.
• SFT of Qwen2.5-7B-Instruct on 4.77M synthetic prompts reaches AIME 24 73.1, AIME 25 65.6, LCB v5 53.4, versus 12.8/8.0/14.7 for the base model.
• On 1,000-prompt difficulty test, Qwen2.5-72B-Instruct accuracy is 18.5% on PromptCoT 2.0 problems versus 21.3-32.3% for other datasets.

Surprising findings

• Removing the EM stage hurt SFT results more than removing the code-start cold-start stage, with the largest drop on HMMT and Codeforces.
• Existing open corpora such as OpenThoughts3 and OpenR1 gave little or no gain for strong Qwen3 thinking models in self-play, possibly because their data was already seen in training.

Limitations

• Evaluation limited to mathematics and programming; multimodal and agentic extension left to future work.
• Self-play gains depend on verifiable rewards (answer matching, unit tests), and unit tests are generated by an LLM.
• Shorter SFT response length may largely reflect the teacher model rather than the prompts.

Experimental setup

Datasets: Codeforces problems (9,221) AoPS math problems (6,365) DeepScaleR OpenCodeReasoning OpenMathReasoning OpenThoughts3 OpenR1 PromptCoT 1.0 corpus LiveCodeBench contest problems before August 2024

Benchmarks: AIME 24 AIME 25 HMMT Feb 25 LiveCodeBench v5 (2408-2502) LiveCodeBench v6 (2502-2505) Codeforces

Metrics: avg@16 accuracy pass@1 Elo rating accuracy of Qwen2.5-72B-Instruct (difficulty) average reasoning tokens NLL MDS embedding distance

Baselines: OpenCodeReasoning OpenMathReasoning OpenThoughts3 OpenR1 PromptCoT 1.0 OpenThoughts3 (GPT) Ring-Lite-2506

Builds on: PromptCoT 1.0 DPO PPO GRPO LiveCodeBench

Closest papers by meaning

• GReaTer: Gradients Over Reasoning Makes Smaller Language Models Strong Prompt Optimizers (ICLR 2025 · similarity 0.92)
• Evolving Prompts In-Context: An Open-ended, Self-replicating Perspective (ICML 2025 · similarity 0.91)
• StrategyLLM: Large Language Models as Strategy Generators, Executors, Optimizers, and Evaluators for Problem Solving (NeurIPS 2024 · similarity 0.91)
• Prompt Sketching for Large Language Models (ICML 2024 · similarity 0.90)
• Improving Factuality and Reasoning in Language Models through Multiagent Debate (ICML 2024 · similarity 0.90)
• Exploring the Limits of Sub-Billion Language Model Reasoners with Open Training Recipes (ICLR 2026 · similarity 0.90)