Research map / Large language models / Agents, code and math
Autoformalization with Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Scale PaLM from 8B to 540B | PaLM autoformalization · MiniF2F algebra | BLEU: 31.49 -> 50.30 | Helped |
| Scale PaLM from 8B to 540B | PaLM autoformalization · MiniF2F number theory | BLEU: 22.10 -> 36.16 | Helped |
| Use Codex instead of PaLM 540B | Autoformalization · MiniF2F algebra | BLEU: 50.30 -> 57.13 | Helped |
| Use Codex instead of PaLM 540B | Autoformalization · MiniF2F number theory | BLEU: 36.16 -> 43.33 | Helped |
| Add one expert iteration using proofs found on autoformalized statements | Thor (M0) · MiniF2F valid | proof success rate: 28.3% -> 36.1% | Helped |
| Add one expert iteration using proofs found on autoformalized statements | Thor (M0) · MiniF2F test | proof success rate: 29.9% -> 34.0% | Helped |
| Add a second expert iteration | Thor after one expert iteration (M1) · MiniF2F valid | proof success rate: 36.1% -> 37.3% | Helped |
| Add a second expert iteration | Thor after one expert iteration (M1) · MiniF2F test | proof success rate: 34.0% -> 35.2% | Helped |
| Use an additional, tangentially related few-shot example that explains how to formalize a line | Codex autoformalization · Linear-function statement case study | formalization correctness | Helped |
| Increase PaLM model size from 8B or 64B to 540B | PaLM autoformalization · Madeline's school rows-of-eight problem | formalization correctness | Helped |
Closest papers by meaning
- Autoformalize Mathematical Statements by Symbolic Equivalence and Semantic Consistency (NeurIPS 2024 · similarity 0.93)
- Don't Trust: Verify -- Grounding LLM Quantitative Reasoning with Autoformalization (ICLR 2024 · similarity 0.90)
- Autoformalizing Euclidean Geometry (ICML 2024 · similarity 0.89)
- ProofAug: Efficient Neural Theorem Proving via Fine-grained Proof Structure Analysis (ICML 2025 · similarity 0.89)
- ATLAS: Autoformalizing Theorems through Lifting, Augmentation, and Synthesis of Data (NeurIPS 2025 · similarity 0.88)
- APOLLO: Automated LLM and Lean Collaboration for Advanced Formal Reasoning (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.