Research map / Large language models / Agents, code and math

RepoAudit: An Autonomous LLM-Agent for Repository-Level Code Auditing

ICML 2025 · Large language models › Agents, code and math · approach: Software engineering agents · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes RepoAudit, an LLM agent that performs autonomous, demand-driven, path-sensitive repository-level bug detection (NPD, MLK, UAF) with a validator module to curb hallucinations.

Problem

Repository-level code auditing with LLMs is hindered by limited context windows, hallucinations that compound across prompts, and high time and token costs when analyzing large codebases for path-sensitive bugs.

Approach

An agent explores the repository function by function, using LLM-driven program abstraction, pointer handling, and feasible path exploration within each function, stores data-flow facts in agent memory, and validates outputs with a control-flow order check and an LLM-based path-condition satisfiability check before reporting bugs.

Key results

• RepoAudit with Claude 3.5 Sonnet detects 40 true bugs across 15 projects with 78.43% precision, averaging 0.44 hours and $2.54 per project.
• Compared with Meta Infer (7 TPs, 2 FPs on eight analyzed projects) and Amazon CodeGuru (0 TPs, 18 FPs), RepoAudit finds substantially more true bugs.
• RepoAudit reports 185 new bugs in ten actively maintained repositories, with 174 confirmed or fixed; precision is 88.46% with DeepSeek R1 and 86.79% with Claude 3.7 Sonnet, and 82.35% with o3-mini.

Measured effects reported in the paper

Surprising findings

• Directly prompting Claude 3.5 Sonnet with all relevant functions produced substantial hallucinations, reporting almost all dereferenced pointers as null even with few-shot examples.
• Meta Infer crashed on five of the thirteen projects that compiled successfully, even across multiple Infer versions.

Limitations

• Relies on LLM hallucination-prone reasoning; the validator only checks control-flow order and path-condition satisfiability, so FPs remain (precision 78.43%).
• Call-context exploration is bounded to four functions (K=4), which may miss deeper inter-procedural bugs.
• Evaluated on C projects and three bug types only; source matchers are hand-implemented per bug type.

Experimental setup

Datasets: 15 real-world C repositories from existing studies (sofa-pbrpc, ImageMagick, coturn, libfreenect, openldap, libsass, mem

Benchmarks: 15 real-world C projects with NPD, MLK, UAF bugs ten actively maintained repositories for new bug discovery

Metrics: precision true positives false positives # prompts input/output tokens financial cost time

Baselines: LLMDFA Meta Infer Amazon CodeGuru single-function LLM detector multiple-function LLM detector

Builds on: LLMDFA tree-sitter chain-of-thought prompting

Closest papers by meaning

• RepoLaunch: Automating Build and Management of Code Repositories across Languages and Platforms (NeurIPS 2026 · similarity 0.90)
• Build-and-Find: An Effort-Aware Protocol for Evaluating Agent-Managed Codebases (NeurIPS 2026 · similarity 0.88)
• SWT-Bench: Testing and Validating Real-World Bug-Fixes with Code Agents (NeurIPS 2024 · similarity 0.88)
• RepoMaster: Autonomous Exploration and Understanding of GitHub Repositories for Complex Task Solving (NeurIPS 2025 · similarity 0.88)
• RepoGraph: Enhancing AI Software Engineering with Repository-level Code Graph (ICLR 2025 · similarity 0.87)
• Improving Code Localization with Repository Memory (ICLR 2026 · similarity 0.87)