Research map / Large language models / Language, safety and interpretability
Viverra: Text-to-Code with Guarantees
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Viverra prompts an LLM to generate a C program plus candidate assertions from a task description, then verifies them compositionally with a portfolio of bounded model checkers and surfaces verified facts as code comments, which a crowdworker user study found improved comprehension.
Problem
LLM-generated code carries no correctness guarantees, so developers must manually review it; the paper seeks practical guarantees between testing and full formal verification.
Approach
Partial co-specification: the LLM co-generates a program and natural-language properties turned into assertions, which are then checked best-effort by bounded model checking with compositional assumptions from the call graph, yielding bounded invariants or conditional facts.
Key results
- Of 17 benchmark rows in the table, 27.9 of ~30 assertions per task on average are verified standalone (mean 27.9 verified, 0.3 conditionally verified, 1.3 unverified).
- base_addition is an outlier with 14 of 23 assertions unverified, attributed to over-approximation by the verifier.
- In a MTurk user study with at least 20 workers per HIT, annotated programs gave higher accuracy on most of 10 questions and participants were faster overall.
Surprising findings
- On one question, annotated-group users were more accurate (100% vs 40%) but took longer reading annotations.
- Verification time is dominated by a few hard global-property assertions, e.g. sudoku took 2077.6 s.
Limitations
- Guarantees only hold up to a bounded loop-unwind depth (k=5).
- Only supports languages and properties the bounded model checkers can handle (C; no heap or quantified invariants).
- User study was small-scale with crowdworkers rather than professional developers.
Experimental setup
Benchmarks: 18 custom text-to-C tasks
Metrics: assertion verification status (verified / conditionally veri LLM and verification time accuracy and response time in user study
Baselines: Daikon Lemur Clover Baldur
Builds on: CBMC ESBMC Bitwuzla Daikon Lemur
Closest papers by meaning
- AlphaVerus: Bootstrapping Formally Verified Code Generation through Self-Improving Translation and Treefinement (ICML 2025 · similarity 0.89)
- Verify0: Can AI Agents Build Formally Verified Software Repositories? (NeurIPS 2026 · similarity 0.86)
- Unsupervised Evaluation of Code LLMs with Round-Trip Correctness (ICML 2024 · similarity 0.86)
- VERSE: Verification-based Self-Play for Code Instructions (AAAI 2025 · similarity 0.86)
- rStar-Coder: Scaling Competitive Code Reasoning with a Large-Scale Verified Dataset (NeurIPS 2025 · similarity 0.85)
- SWT-Bench: Testing and Validating Real-World Bug-Fixes with Code Agents (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.