Research map / Large language models / Reasoning and chain of thought
Pessimistic Verification for Open-Ended Math Questions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes pessimistic verification (reject a proof if any of several parallel verifiers flags an error), with a progressive proof-halving variant, and shows gains in verification accuracy and token efficiency over standard verification and long CoT, plus an agentic solving workflow on IMO 2025 and MathArena Apex 2025.
Problem
Automatic verification of open-ended math proofs is weak in error detection, generalizability and cost-efficiency, which limits math-solving agents and RL reward signals.
Approach
Treat verification as error detection: run multiple parallel verifiers and reject a solution if any one finds a flaw. The progressive variant recursively halves the proof into chunks, verifying at multiple scales and pruning proofs already flagged, so detailed checking is spent only where needed.
Key results
- On IMO-GradingBench subset, GPT-5-mini progressive@3/6 raises Bal. F1 from 0.780 (standard) to 0.849, with TNR rising from 0.675 to 0.842.
- On IMO 2025 with Gemini 3 Pro Preview, prog@4/4 scores 22 total versus 13 for direct solving, at about 985K equivalent tokens versus 1,310K for pes@12.
- Manual review finds that most FN cases from GPT-5-mini or stronger models identify real critical errors missed by dataset annotations.
Surprising findings
- Existing verification benchmarks appear to underestimate stronger models: many apparent false negatives are real errors missed by annotators.
- Majority voting is nearly ineffective for verification, while pessimistic verification keeps improving with more samples.
Limitations
- Benchmark labels are unreliable for strong models, so reported gains on stronger models are hard to measure precisely.
- Efficiency advantage relies on the cost-weighting convention (output plus input/8 or /6) chosen by the authors.
- Problem-solving experiments use only 6 IMO problems and 12 Apex problems with small trial counts.
Experimental setup
Datasets: IMO-GradingBench Hard2Verify QiuZhen-Bench IMO 2025 MathArena Apex 2025
Benchmarks: IMO-GradingBench Hard2Verify QiuZhen-Bench IMO 2025 MathArena Apex 2025
Metrics: True Negative Rate True Positive Rate Balanced F1 accuracy equivalent output tokens
Baselines: standard verification simple pessimistic verification long CoT majority voting DeepSeekMath-V2 workflow
Builds on: chain-of-thought majority voting
Closest papers by meaning
- Step-by-Step Reasoning for Math Problems via Twisted Sequential Monte Carlo (ICLR 2025 · similarity 0.86)
- Escaping the Cognitive Well: Efficient Competition Math with Off-the-Shelf Models (NeurIPS 2026 · similarity 0.86)
- NFT: Bridging Supervised Learning and Reinforcement Learning in Math Reasoning (ICLR 2026 · similarity 0.86)
- V1: Unifying Generation and Self-Verification for Parallel Reasoners (ICML 2026 · similarity 0.86)
- FaithSieve: Fine-Grained Evaluation of Math Proofs with Faithful Formal Evidence (NeurIPS 2026 · similarity 0.85)
- ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.