Research map / Large language models / Agents, code and math
NaturalProver: Grounded Mathematical Proof Generation with Language Models
Sean Welleck, Jiacheng Liu, Ximing Lu, Hannaneh Hajishirzi, Yejin Choi
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Condition proof generation on human-provided reference titles | GPT-3 proof generation · NaturalProofs-Gen core test, full proof | overall usefulness: 20% -> 35% | Improved |
| Condition proof generation on retrieved reference titles | GPT-3 proof generation · NaturalProofs-Gen core test, full proof | overall correctness: 13% -> 24% | Improved |
| Use stepwise++ decoding with provided references | NaturalProver · NaturalProofs-Gen core test, full proof | overall usefulness: 35% -> 45% | Improved |
| Add reference reconstruction during training (ablation) | NaturalProver with provided references · NaturalProofs-Gen core validation, full proof | Ref-F1: 82.85 -> 84.15 | Improved |
| Add reference reconstruction during training (ablation) | NaturalProver with provided references · NaturalProofs-Gen core validation, full proof | GLEU: 33.03 -> 35.93 | Improved |
| Add multi-temperature expansion (ablation) | Stepwise++ decoding · NaturalProofs-Gen core dev, full proof | Ref-F1: 93.11 (.34) -> 94.13 (.45) | Improved |
| Add selection across different constraint-weight settings (ablation) | Stepwise++ decoding · NaturalProofs-Gen core dev, full proof | Ref-F1: 92.61 (.63) -> 94.13 (.45) | Improved |
| Use stepwise++ rather than reranking 10 full proofs (ablation) | NaturalProver decoding · NaturalProofs-Gen core dev, full proof | Ref-F1: 91.72 (.28) -> 94.13 (.45) | Improved |
| Use reranking of 10 full proofs rather than stepwise++ (ablation) | NaturalProver decoding · NaturalProofs-Gen core dev, full proof | GLEU: 40.11 (1.55) -> 43.88 (.29) | Improved |
| Increase constraint weight α from 0.0 to 1.0 (ablation) | NaturalProver value function · NaturalProofs-Gen core dev, full proof | Ref-F1: 88.40 -> 93.60 | Improved |
| Increase constraint weight α from 0.0 to 1.0 (ablation) | NaturalProver value function · NaturalProofs-Gen core dev, full proof | GLEU: 42.79 -> 41.90 | Hurt |
| Increase GPT-2 inference-time compute by reranking 10 proofs (ablation) | NaturalProver with GPT-2 · NaturalProofs-Gen core dev, full proof | Ref-F1: 65.22 -> 89.30 | Improved |
Closest papers by meaning
- LEGO-Prover: Neural Theorem Proving with Growing Libraries (ICLR 2024 · similarity 0.91)
- ProofAug: Efficient Neural Theorem Proving via Fine-grained Proof Structure Analysis (ICML 2025 · similarity 0.90)
- Lean-STaR: Learning to Interleave Thinking and Proving (ICLR 2025 · similarity 0.90)
- FaithSieve: Fine-Grained Evaluation of Math Proofs with Faithful Formal Evidence (NeurIPS 2026 · similarity 0.90)
- miniCTX: Neural Theorem Proving with (Long-)Contexts (ICLR 2025 · similarity 0.89)
- Alchemy: Amplifying Theorem-Proving Capability Through Symbolic Mutation (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
