Research map / Large language models / Agents, code and math
Verifier-Backed Hard Problem Generation for Mathematical Reasoning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes VHG, a three-party self-play framework where a verifier gates validity before solver failure counts as setter reward, instantiated with a SymPy hard verifier for indefinite integrals and an LLM-judge soft verifier for general math.
Problem
Self-play problem generation for math rewards setters with solver failure, which can be hacked by generating invalid or underspecified problems; the goal is to generate valid, novel, and hard math problems for training solvers and challenge sets.
Approach
Adds an independent verifier to the setter-solver game so the setter's reward (1 - solver accuracy) is only granted when the verifier accepts the problem-reference pair; uses SymPy differentiation checks for integrals and an LLM judge plus rule filters for general math.
Key results
- On indefinite integral, VHG (Hard) raises Pass@1 from 28.8% to 45.4% on Competition, 52.5% to 69.4% on Qualifier, and 43.3% to 64.7% on Integration Stress Test, versus R-Zero iterations at about 30-32% on Competition.
- On general math, VHG (Soft) improves overall Pass@1 from 56.79% (Qwen3-4B-Base) to 69.01%, above Vanilla-GRPO (67.62%) and R-Zero iter. 1-3 (about 65.6-66.2%).
- Challenge pools with zero local pass rate remain hard for Qwen3-32B, with Pass@1 of 46.99% and Pass@8 of 85.99% on indefinite integral challenge problems.
Surprising findings
- R-Zero does not outperform vanilla GRPO on the existing dataset after three iterations, and its Pass@1 on integral benchmarks drops below the base model's Competition score.
- Verifier-valid generated problems that are unsolved by the 4B solver still challenge Qwen3-32B, suggesting weak-to-strong data generation.
Limitations
- Hard verifier only applies to tasks with symbolic checks such as indefinite integrals; general math relies on a noisier LLM judge.
- General math validity is judge-based rather than exact, and the general math soft verifier underperforms on GSM8K relative to R-Zero.
- Experiments use a single 4B backbone and limited seed sets; the evaluation scale is modest.
Experimental setup
Datasets: MATH GSM8K AntiderivBench Integration Stress Test (curated, 532 items) college textbook integration problems
Benchmarks: AntiderivBench Qualifier AntiderivBench Competition Integration Stress Test MATH GSM8K AMC Minerva Olympiad AIME24 AIME25 AIME26
Metrics: pass@1 pass@8
Baselines: Vanilla GRPO R-Zero Seed-only Seed + Cold-SFT
Builds on: R-Zero GRPO Absolute Zero MathSmith
Closest papers by meaning
- MathSmith: Towards Extremely Hard Mathematical Reasoning by Forging Synthetic Problems with a Reinforced Policy (AAAI 2026 · similarity 0.90)
- MathFimer: Enhancing Mathematical Reasoning by Expanding Reasoning Steps through Fill-in-the-Middle Task (ICLR 2026 · similarity 0.89)
- Frugal LMs Trained to Invoke Symbolic Solvers Achieve Parameter-Efficient Arithmetic Reasoning (AAAI 2024 · similarity 0.89)
- MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoning (ICLR 2024 · similarity 0.89)
- OMEGA: Can LLMs Reason Outside the Box in Math? Evaluating Exploratory, Compositional, and Transformative Generalization (NeurIPS 2025 · similarity 0.89)
- Improving Rationality in the Reasoning Process of Language Models through Self-playing Game (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.