Research map / Large language models / Agents, code and math
Draft, Sketch, and Prove: Guiding Formal Theorem Provers with Informal Proofs
Albert Qiaochu Jiang, Sean Welleck, Jin Peng Zhou, Timothee Lacroix, Jiacheng Liu, Wenda Li, Mateja Jamnik, Guillaume Lample, Yuhuai Wu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Guide Sledgehammer with formal sketches derived from human informal proofs | Sledgehammer + heuristics · miniF2F-test | success rate: 20.9% -> 39.3% | Improved |
| Guide the prover with sketches derived from Minerva 62B informal drafts | Sledgehammer + heuristics · miniF2F-valid | success rate: 18.0% -> 43.9% | Improved |
| Add 11 heuristic tactics | Sledgehammer · miniF2F-test | success rate: 10.4% -> 20.9% | Improved |
| Use Minerva 62B rather than Minerva 8B for informal proof drafting | DSP · miniF2F-test | success rate: 35.3% -> 37.7% | Improved |
| Use Minerva 540B rather than Minerva 62B for informal proof drafting | DSP · miniF2F-test | success rate: 37.7% -> 38.9%; 43.9% -> 42.6% | Mixed |
| Add in-line comments aligning informal proof segments with sketch code (ablation) | DSP · miniF2F-valid | success rate: 4.9% | Improved |
| Add in-line comments aligning informal proof segments with sketch code (ablation) | DSP · miniF2F-test | success rate: 2.8% | Improved |
| Add informal proof drafts before sketching (ablation) | DSP · miniF2F-valid | success rate: 3.7% | Improved |
| Add informal proof drafts before sketching (ablation) | DSP · miniF2F-test | success rate: 5.3% | Improved |
| Use automated provers to close open conjectures in sketches (ablation) | DSP · miniF2F-valid | success rate: 9.8% | Improved |
| Use automated provers to close open conjectures in sketches (ablation) | DSP · miniF2F-test | success rate: 9.0% | Improved |
| Draft informal proofs with Minerva 540B rather than use human-written drafts | DSP with 200 sketch attempts per problem · miniF2F | problems solved: 203 -> 209 | Improved |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- Thor — Thor: Wielding Hammers to Integrate Language Models and Automated Theorem Provers (NeurIPS 2022) · better on 2 of 2 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- ImProver: Agent-Based Automated Proof Optimization (ICLR 2025 · similarity 0.89)
- Reviving DSP for Advanced Theorem Proving in the Era of Reasoning Models (NeurIPS 2025 · similarity 0.89)
- Subgoal-based Demonstration Learning for Formal Theorem Proving (ICML 2024 · similarity 0.88)
- Not All Proofs Are Equal: Evaluating LLM Proof Quality Beyond Correctness (NeurIPS 2026 · similarity 0.88)
- Proving Theorems Recursively (NeurIPS 2024 · similarity 0.88)
- FaithSieve: Fine-Grained Evaluation of Math Proofs with Faithful Formal Evidence (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
