Research map / Multi-agent systems and games / LLM agents
Dynamic Risk Assessments for Offensive Cybersecurity Agents
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Argues that offensive cybersecurity agent risk assessments should account for adversary compute-driven self-improvement, and shows that five degrees of freedom (repeated sampling, more interaction rounds, iterative prompt refinement, self-training, iterative workflow refinement) raise InterCode CTF pass@k by over 40% within an 8 H100-hour budget.
Problem
Static evaluations of offensive cybersecurity agents ignore how adversaries can spend compute to modify and iteratively improve agents, so assessed risk may be underestimated.
Approach
Defines an expanded threat model with five adversary degrees of freedom (deployment-time and adaptation-time) using only internal verifier feedback, and compares their performance-cost trade-offs under a fixed compute budget in stateful vs. non-stateful environments.
Key results
- Under an 8 H100 GPU-hour budget (under $36 of compute), the best configuration improves InterCode CTF performance by more than 40% relative to the baseline pass@1 of 0.58.
- Iterative prompt refinement reduces the average number of failed InterCode CTF tasks from 16.20 to 6.70 across iterations, vs. 15.08 to 7.29 for repeated sampling with bootstrap estimation.
- Self-training on 33 successful dev trajectories (5 epochs) generalizes in-domain, consistently outperforming the base model across k on the InterCode CTF test set.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase repeated samples k from 1 to 10 (ablation) | NYU CTF Agent with Qwen2.5-32B-Coder-Instruct · InterCode CTF (Test), NYU CTF (Test), and Cybench | pass@k | Helped |
| Increase max interaction rounds N from 10 to 30 (ablation) | NYU CTF Agent with Qwen2.5-32B-Coder-Instruct · InterCode CTF (Test), NYU CTF (Test), and Cybench | pass@k | Helped |
| Refine the initial user prompt iteratively using failed trajectories | NYU CTF Agent with Qwen2.5-32B-Coder-Instruct · InterCode CTF (Test) | pass@k | Helped |
| Fine-tune on successful self-generated trajectories for 5 epochs | Qwen2.5-32B-Coder-Instruct in the NYU CTF Agent · InterCode CTF (Test) | pass@k | Helped |
| Increase self-training from 5 to 10 epochs (ablation) | Qwen2.5-32B-Coder-Instruct in the NYU CTF Agent · InterCode CTF (Dev) at pass@1; InterCode CTF (Dev and Test) at larger k | pass@k | Mixed |
| Refine the agent workflow iteratively using development-set performance | NYU CTF Agent with Qwen2.5-32B-Coder-Instruct · InterCode CTF (Dev and Test) | pass@1 | Helped |
| Use iterative prompt refinement instead of repeated sampling | NYU CTF Agent with Qwen2.5-32B-Coder-Instruct · InterCode CTF (Test) | failed tasks: from 16.20 to 6.70 | Helped |
| Use iterative prompt refinement instead of repeated sampling | NYU CTF Agent with Qwen2.5-32B-Coder-Instruct · InterCode CTF (Test) | format mismatch failures: 2.90 to 0.20 | Helped |
| Use iterative prompt refinement instead of repeated sampling | NYU CTF Agent with Qwen2.5-32B-Coder-Instruct · InterCode CTF (Test) | max rounds exceeded failures: 7.3 to 3.1 | Helped |
| Use iterative prompt refinement instead of repeated sampling | NYU CTF Agent with Qwen2.5-32B-Coder-Instruct · InterCode CTF (Test) | tunnel vision failures: 0.9 to 0.6 | Helped |
| Increase max interaction rounds N from 20 to 40 (ablation) | NYU CTF Agent with Qwen2.5-32B-Coder-Instruct · InterCode CTF (Test), stateful environment | pass@1 | Helped |
| Increase max interaction rounds beyond N=40 (ablation) | NYU CTF Agent with Qwen2.5-32B-Coder-Instruct · InterCode CTF (Test), stateful environment | pass@1 | No clear effect |
Surprising findings
- Self-training on only 33 trajectories still yields in-domain generalization to the test set.
- Simple iterative prompt refinement outperforms repeated sampling, so pass@k based on repeated sampling underestimates risk.
Limitations
- Only a small compute budget (8 H100 GPU hours) and a single core model and scaffold were evaluated.
- Excludes external assistance by design, so results do not capture risk from external knowledge or stronger models.
- Some InterCode CTF tasks were excluded as broken (10 of 100) and the test set is small (36 tasks).
Experimental setup
Datasets: InterCode CTF NYU CTF Cybench
Benchmarks: InterCode CTF (Test) NYU CTF (Test) Cybench
Metrics: pass@k pass@1 failure mode frequency
Baselines: NYU CTF Agent base scaffolding with Qwen2.5-32B-Coder-Instruct repeated sampling
Builds on: NYU CTF Agent STaR ADAS InterCode
Closest papers by meaning
- CyberGym: Evaluating AI Agents' Real-World Cybersecurity Capabilities at Scale (ICLR 2026 · similarity 0.88)
- A New Framework for Cybersecurity Refusals in AI Agents (ICML 2026 · similarity 0.88)
- Cybench: A Framework for Evaluating Cybersecurity Capabilities and Risks of Language Models (ICLR 2025 · similarity 0.88)
- ORBIT: A Framework for Multi-Agent Security Evaluations (NeurIPS 2026 · similarity 0.86)
- Agent Security is a Systems Problem (NeurIPS 2026 · similarity 0.85)
- CVE-Bench: A Benchmark for AI Agents’ Ability to Exploit Real-World Web Application Vulnerabilities (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.