Research map / Large language models / Agents, code and math
Thor: Wielding Hammers to Integrate Language Models and Automated Theorem Provers
Albert Qiaochu Jiang, Wenda Li, Szymon Tworkowski, Konrad Czechowski, Tomasz Odrzygóźdź, Piotr Miłoś, Yuhuai Wu, Mateja Jamnik
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| offload premise selection to Sledgehammer via a learned <hammer> token (Thor) instead of a plain language model | 700M language model · PISA/test | success rate (%): 39.0 -> 57.0 | Improved |
| integrate hammer inside the proof search instead of taking the union of separate LM and Sledgehammer results | Thor · PISA/test | success rate (%): 48.8 -> 57.0 | Improved |
| use Thor on MiniF2F instead of the plain language model | 700M language model · MiniF2F test | success rate (%): 24.2 -> 29.9 | Improved |
| train the model to also learn how to select premises (no <hammer> replacement) (ablation) | Thor · PISA/test | success rate (%): 57.0 -> 55.4 | Hurt |
| add most recent proof step as context in the input (ablation) | Thor · PISA/test | success rate (%): 53.6 -> 57.0 | Improved |
| raise sampling temperature from 1.0 to 1.2 (ablation) | Thor · PISA/test | success rate (%): 55.7 -> 57.0 | Improved |
| Thor vs expert iteration at fraction of the compute | language model for theorem proving · MiniF2F test | success rate (%): 29.6 -> 29.9 | Improved |
Later papers that compared with this method
3 later accepted papers report Thor as a baseline in their results tables; 3 of them report a better number on most of the results they share with it.
- Subgoal-based Demonstration Learning for Formal Theorem Proving (ICML 2024 · better on 2 of 2 shared results)
- Magnushammer: A Transformer-Based Approach to Premise Selection (ICLR 2024 · better on 1 of 1 shared result)
- Draft, Sketch, and Prove: Guiding Formal Theorem Provers with Informal Proofs (ICLR 2023 · better on 2 of 2 shared results)
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Magnushammer: A Transformer-Based Approach to Premise Selection (ICLR 2024 · similarity 0.92)
- LEGO-Prover: Neural Theorem Proving with Growing Libraries (ICLR 2024 · similarity 0.89)
- MUSTARD: Mastering Uniform Synthesis of Theorem and Proof Data (ICLR 2024 · similarity 0.89)
- MPS-Prover: Advancing Stepwise Theorem Proving by Multi-Perspective Search and Data Curation (NeurIPS 2025 · similarity 0.89)
- ProofAug: Efficient Neural Theorem Proving via Fine-grained Proof Structure Analysis (ICML 2025 · similarity 0.89)
- Lean-STaR: Learning to Interleave Thinking and Proving (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
