AI Paper Atlas

Research map / Large language models / Agents, code and math

Scaling Generative Verifiers For Natural Language Mathematical Proof Verification And Selection

Sadegh Mahdavi, Branislav Kisacanin, Shubham Toshniwal, Wei Du, Ivan Moshkov, George Armstrong, Renjie Liao, Christos Thrampoulidis, Igor Gitman

ICML 2026 · Large language models › Agents, code and math · approach: Retrieval-augmented formal theorem proving · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
a simple MLP on text embeddings, with no math checking, exploits dataset imbalanceVerOPC test set · VerOPCaccuracy (%): 75.34% accuracy, more than 5% above Claude-Sonnet-4Improved
use RL (GRPO) to train the judge (ablation)Qwen3-30B-A3B-Thinking-2507 LLM-as-a-Judge · VerProofArena, OPC promptF1: 37.23 -> 68.75Improved
add RL training to the judge (ablation)LLM-as-a-Judge prompts · VerProofArenaprompt sensitivity (F1 gap across prompts): OPC 68.75 vs GIMO 65.40 after RLImproved
add RL training to the judge (ablation)OPC-prompt judge · Challenge-19 final-answer precisionprecision: 60.41 -> 62.59No clear effect
add rubric at test time to a model trained without rubric (ablation)OPC RL judge · VerProofArena and Challenge-19F1 and final-answer precision: F1 68.75 -> 73.35; precision 62.59 -> 99.31Improved
add rubric or ground-truth proof to the judge prompt in both training and evaluation (ablation)OPC RL judge · VerProofArenaF1: 68.75 -> 56.69Hurt
add rubric or ground-truth proof to the prompt (ablation)LLM-as-a-Judge and GenSelect · SelProofBench, SelOPC proof selectionselection accuracyNo clear effect
replace binary grading prompt with 7-point grading (ablation)LLM-as-a-Judge · SelProofBench, SelOPCselection accuracyNo clear effect
ensemble different judges by averaging scores (ablation)LLM-as-a-Judge · SelOPCselection accuracyNo clear effect
use step-based judgement with all-steps-correct aggregation (ablation)LLM-as-a-Judge · proof verificationaccuracy and recallHurt
increase number of sampled judgements per proof up to 32 (ablation)LLM-as-a-Judge · proof selectionselection accuracyImproved
use hybrid knockout GenSelect plus LLM-as-a-Judge instead of majority votingGPT-OSS-120B · Challenge-19accuracy (%): 82.23 -> 96.05Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy