Research map / Large language models / Agents, code and math
VeriEquivBench: An Equivalence Score for Ground-Truth-Free Evaluation of Formally Verifiable Code
Lingfei Zeng, Fengdi Che, Xuhan Huang, Fei Ye, Xu Xu, Binhang Yuan, Jie Fu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Dafny-verifier equivalence score instead of NL-based Clover check on DafnySynthesis | evaluation metric · DafnySynthesis | validation: 10% -> 10% plus 9 more flagged | Improved |
| evaluate on TagComp instead of CloverBench | claude-4-sonnet · end-to-end verifiable code gen pass@4 | success rate: 75.81% -> 0 | Hurt |
| supply two examples in specification-generation prompt (ablation) | Claude-4-sonnet spec generation · LeetCode autoformalization | syntax error rate | Improved |
| use multi-stage code generation with a lighter polishing model | Dafny code generation · LeetCode | verifiable rate | Improved |
| use RL-trained baseline for spec generation | SFT model · validation set | equivalence score | No clear effect |
Closest papers by meaning
- AlphaVerus: Bootstrapping Formally Verified Code Generation through Self-Improving Translation and Treefinement (ICML 2025 · similarity 0.90)
- How efficient is LLM-generated code? A rigorous & high-standard benchmark (ICLR 2025 · similarity 0.88)
- CLEVER: A Curated Benchmark for Formally Verified Code Generation (NeurIPS 2025 · similarity 0.88)
- FVEL: Interactive Formal Verification Environment with Large Language Models via Theorem Proving (NeurIPS 2024 · similarity 0.88)
- Verify0: Can AI Agents Build Formally Verified Software Repositories? (NeurIPS 2026 · similarity 0.88)
- Beyond Accuracy: Evaluating Self-Consistency of Code Large Language Models with IdentityChain (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
