Research map / Multi-agent systems and games / LLM agents

Is Vibe Coding Safe? Benchmarking Vulnerability of Agent-Generated Code in Real-World Tasks

ICML 2026 · Multi-agent systems and games › LLM agents · approach: Software engineering agent benchmarks · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces SusVibes, a 186-task repository-level benchmark built from real human-introduced vulnerabilities to test whether coding agents' feature implementations are both functionally correct and secure, finding that most functionally correct solutions are insecure.

Problem

Vibe coding agents are increasingly used to implement features in real repositories, but existing benchmarks focus on functional correctness (SWE-bench) or small-scope secure code generation; it is unclear whether agent-generated feature code is secure.

Approach

Build security-sensitive feature-request tasks by masking the implementation of features that were once vulnerable (from vulnerability-fix commits), writing issue-style descriptions without security hints, and evaluating with human-written functional and security tests in Dockerized environments.

Key results

• SWE-agent with Claude 4 Sonnet passes functional tests on 57.0% of tasks but only 11.8% are both functionally correct and secure (about 79% of correct solutions are insecure).
• Across 12 agent settings (3 frameworks x 4 LLMs), SecPass ranges roughly 3.8% to 12.9%, averaging around 10%.
• Self-selection of CWE risks lowers FuncPass from 57.0 to 50.0 while raising SecPass only from 11.8 to 14.5; oracle CWE hints give 15.1 SecPass with FuncPass 55.4.

Surprising findings

• Providing oracle vulnerability hints barely improves security (SecPass 11.8 to 15.1) while reducing functional correctness.
• Gemini 3 Pro greatly improves functional correctness over Gemini 2.5 Pro but its security gains are limited.

Limitations

• Benchmark is limited to 100 Python repositories and 186 tasks.
• Security tests are human-written and tied to specific vulnerabilities, so they cover only a subset of security properties.
• Only preliminary prompting-based mitigations were explored.

Experimental setup

Datasets: ReposVul MoreFixes

Benchmarks: SusVibes

Metrics: FuncPass@1 SecPass@1 SecPass restricted to functionally correct subset

Baselines: SWE-agent OpenHands Claude Code

Builds on: SWE-bench SWE-agent OpenHands ReposVul MoreFixes BaxBench

Closest papers by meaning

• MOSAIC-Bench: Measuring Compositional Vulnerability Induction in Coding Agents (NeurIPS 2026 · similarity 0.86)
• SEC-bench: Automated Benchmarking of LLM Agents on Real-World Software Security Tasks (NeurIPS 2025 · similarity 0.85)
• Shadows in the Code: Exploring the Risks and Defenses of LLM-based Multi-Agent Software Development Systems (AAAI 2026 · similarity 0.85)
• Training Language Models to Generate Quality Code with Program Analysis Feedback (NeurIPS 2025 · similarity 0.84)
• RedCode: Risky Code Execution and Generation Benchmark for Code Agents (NeurIPS 2024 · similarity 0.84)
• CVE-Factory: Scaling Expert-Level Agentic Tasks for Code Security Vulnerability (ICML 2026 · similarity 0.83)