Research map / Large language models / Agents, code and math
StepFun-Formalizer: Unlocking the Autoformalization Potential of LLMs Through Knowledge-Reasoning Fusion
Yutong Wu, Di Huang, Ruosi Wan, Yue Peng, Shijie Shang, Chenrui Cao, Lei Qi, Rui Zhang, Xishan Zhang, Zidong Du, Jie Yan, Xing Hu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add stage-1 formal knowledge SFT data (ablation) | StepFun-Formalizer-7B · ProverBench | BEq@1 (%): 24.5 -> 25.1 | Improved |
| add stage-1 formal knowledge SFT data (ablation) | StepFun-Formalizer-7B · CombiBench | BEq@1 (%): 3.9 -> 5.2 | Improved |
| add informal-to-formal reasoning SFT data (ablation) | StepFun-Formalizer-7B · ProverBench | BEq@16 (%): 25.3 -> 37.9 | Improved |
| add informal-to-formal reasoning SFT data (ablation) | StepFun-Formalizer-7B · ProverBench | BEq@1 (%): 21.8 -> 25.1 | Improved |
| add informal-to-formal reasoning SFT data (ablation) | StepFun-Formalizer-7B · CombiBench | BEq@1 (%): 5.3 -> 5.2 | Hurt |
| use expert template-guided reasoning instead of direct distillation from Claude4-thinking (ablation) | StepFun-Formalizer-7B · ProverBench | BEq@1 (%): 21.8 -> 25.1 | Improved |
| use expert template-guided reasoning instead of direct distillation from Claude4-thinking (ablation) | StepFun-Formalizer-7B · ProverBench | BEq@16 (%): 33.3 -> 37.9 | Improved |
| add LLM-based validity selection of knowledge data (ablation) | StepFun-Formalizer-7B · ProverBench | BEq@16 (%): 35.6 -> 37.9 | No clear effect |
| add LLM-based validity selection of knowledge data (ablation) | StepFun-Formalizer-7B · ProverBench | BEq@1 (%): 25.4 -> 25.1 | No clear effect |
| start from DeepSeek-R1-Distill-Qwen-7B instead of Kimina-Autoformalizer-7B (ablation) | StepFun-Formalizer-7B · ProverBench | BEq@1 (%): 21.4 -> 25.1 | Improved |
| majority voting over 16 samples | Kimina-Autoformalizer-7B · FormalMATH-Lite | BEq (%): 35.1 -> 45.9 | Improved |
| add RLVR with BEq reward after SFT (ablation) | StepFun-Formalizer-7B · average of benchmarks | average BEq@1: 0.258 -> 0.303 | Improved |
Closest papers by meaning
- Autoformalize Mathematical Statements by Symbolic Equivalence and Semantic Consistency (NeurIPS 2024 · similarity 0.91)
- MathFimer: Enhancing Mathematical Reasoning by Expanding Reasoning Steps through Fill-in-the-Middle Task (ICLR 2026 · similarity 0.90)
- ATLAS: Autoformalizing Theorems through Lifting, Augmentation, and Synthesis of Data (NeurIPS 2025 · similarity 0.90)
- Don't Trust: Verify -- Grounding LLM Quantitative Reasoning with Autoformalization (ICLR 2024 · similarity 0.90)
- Putnam-AXIOM: A Functional & Static Benchmark for Measuring Higher Level Mathematical Reasoning in LLMs (ICML 2025 · similarity 0.89)
- APOLLO: Automated LLM and Lean Collaboration for Advanced Formal Reasoning (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
