Research map / Large language models / Agents, code and math
AlgoVeri: An Aligned Benchmark for Verified Code Generation on Classical Algorithms
Haoyu Zhao, Ziran Yang, Jiawei Li, Mike He, Zenan Li, Chi Jin, Venugopal Veeravalli, Aarti Gupta, Sanjeev Arora
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase parallel sampling from 1 to 10 chains of 15 repair rounds (ablation) | GPT-OSS-120B · AlgoVeri Dafny | full mark pass rate (%): 13.51 -> 28.57 | Improved |
| increase parallel sampling from 1 to 10 chains of 15 repair rounds (ablation) | Qwen3-235B-A22B-Instruct · AlgoVeri Dafny | full mark pass rate (%): 18.31 -> 29.87 | Improved |
| increase parallel sampling from 1 to 10 chains of 15 repair rounds (ablation) | Qwen3-Next-80B-A3B-Thinking · AlgoVeri Dafny | full mark pass rate (%): 17.27 -> 33.77 | Improved |
| increase parallel sampling from 1 to 10 chains of 15 repair rounds (ablation) | Devstral-2-123B-Instruct · AlgoVeri Dafny | full mark pass rate (%): 6.10 -> 18.18 | Improved |
| increase parallel sampling from 1 to 10 chains of 15 repair rounds (ablation) | GPT-OSS-120B · AlgoVeri Lean | full mark pass rate (%): 7.01 -> 14.29 | Improved |
| increase parallel sampling from 1 to 10 chains of 15 repair rounds (ablation) | Qwen3-235B-A22B-Instruct · AlgoVeri Verus | full mark pass rate (%): 9.09 -> 12.99 | Improved |
| add semantic filtering by an LLM judge to compiler-verified results | Gemini-3 Flash · AlgoVeri Dafny | pass rate (%): 55.84 -> 40.26 | Hurt |
| add semantic filtering by an LLM judge to compiler-verified results | GPT-5 mini · AlgoVeri Dafny | pass rate (%): 41.56 -> 30.47 | Hurt |
| add semantic filtering by an LLM judge to compiler-verified results | GPT-5.3 Codex · AlgoVeri Lean | pass rate (%): 23.38 -> 11.69 | Hurt |
| add iterative repair rounds with compiler feedback (ablation) | Gemini-3 Flash · AlgoVeri Dafny | average pass rate: nearly triples from Round 0 to 15 | Improved |
| add iterative repair rounds with compiler feedback (ablation) | GPT-OSS-120B · AlgoVeri | average pass rate: saturates by Round 3 | No clear effect |
| replace pure sampling with repair chains at equal total budget (N/15 chains of 15 rounds) (ablation) | GPT-OSS-120B · AlgoVeri | pass rate | Hurt |
Closest papers by meaning
- AlphaVerus: Bootstrapping Formally Verified Code Generation through Self-Improving Translation and Treefinement (ICML 2025 · similarity 0.91)
- CLEVER: A Curated Benchmark for Formally Verified Code Generation (NeurIPS 2025 · similarity 0.91)
- Verify0: Can AI Agents Build Formally Verified Software Repositories? (NeurIPS 2026 · similarity 0.90)
- Variation in Verification: Understanding Verification Dynamics in Large Language Models (ICLR 2026 · similarity 0.87)
- Towards General Loop Invariant Generation: A Benchmark of Programs with Memory Manipulation (NeurIPS 2024 · similarity 0.87)
- Verifiers in the Loop: Decoding Time Verification for Code Translation (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
