Research map / Large language models / Agents, code and math
Can LLMs Reason Like Automated Theorem Provers for Rust Verification? VCoT-Bench: Evaluating via Verification Chain of Thought
Zichen Xie, Wenxi Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add thinking mode (ablation) | Qwen 3 8B · VCoT-Bench-Type (Invariant removal) | overall accuracy (%): 26.87 -> 10.03 | Hurt |
| add thinking mode (ablation) | Qwen 3 8B · VCoT-Bench-Type (Assertion removal) | overall accuracy (%): 13.70 -> 11.47 | Hurt |
| use Gemini 3.1 Pro instead of Gemini 3 Flash | Gemini · VCoT-Bench-Loc (Middle) | overall accuracy (%): 47.69 -> 36.15 | Hurt |
| move missing blocks from Front to Middle | Claude Sonnet 4.5 · VCoT-Bench-Loc | overall accuracy (%): 69.04 -> 46.35 | Hurt |
| move missing blocks from Front to Middle | Gemini 3.1 Pro · VCoT-Bench-Loc | overall accuracy (%): 62.69 -> 36.15 | Hurt |
| increase removed proof blocks from 10% to 100% (ablation) | Claude Sonnet 4.5 · VCoT-Bench-Ratio | overall accuracy (%): 71.58 -> 17.22 | Hurt |
| increase removed proof blocks to 100% (ablation) | Qwen 3 think · VCoT-Bench-Ratio | overall accuracy (%): 0.66 | Hurt |
| use Best-of-10 sampling instead of greedy decoding | GPT-5.2 · VCoT-Bench-Type (Assertion removal) | overall accuracy (%): 34.25 -> 45.03 (+10.78) | Improved |
| use Best-of-10 sampling instead of greedy decoding | DeepSeek V3.2 · VCoT-Bench-Type (Invariant removal) | overall accuracy (%): 49.49 -> 50.00 (+0.51) | Improved |
| use GPT-5.2 instead of Claude Sonnet 4.5 for Z3-to-Verus transformation (ablation) | VCoT-Lift · 20 Verus-Bench programs | avg lifted proof lines: 7 -> 45 | Improved |
| use Gemini 3.1 Pro instead of GPT-5.2 for proof repair (ablation) | VCoT-Lift · 20 Verus-Bench programs | avg repair rounds: 13 -> 4 | Improved |
Closest papers by meaning
- APOLLO: Automated LLM and Lean Collaboration for Advanced Formal Reasoning (NeurIPS 2025 · similarity 0.87)
- Automated Proof Generation for Rust Code via Self-Evolution (ICLR 2025 · similarity 0.87)
- Lemur: Integrating Large Language Models in Automated Program Verification (ICLR 2024 · similarity 0.87)
- FVEL: Interactive Formal Verification Environment with Large Language Models via Theorem Proving (NeurIPS 2024 · similarity 0.86)
- CoRe: Benchmarking LLMs’ Code Reasoning Capabilities through Static Analysis Tasks (NeurIPS 2025 · similarity 0.86)
- rStar-Coder: Scaling Competitive Code Reasoning with a Large-Scale Verified Dataset (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
