Research map / Large language models / Agents, code and math
ABC-Bench: An Agentic Bio-Capabilities Benchmark for Biosecurity
Andrew Bo Liu, Samira Nedungadi, Bryce Cai, Alex Kleinman, Harmon Bhasin, Seth Donoughe
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| compare frontier models to human expert baseliners on Fragment Design | frontier LLM agents · ABC-Bench Fragment Design | mean score: 0.33 ± 0.12 -> 1.00 ± 0.00 | Improved |
| compare frontier models to human baseliners on Liquid Handling Robot | frontier LLM agents · ABC-Bench Liquid Handling Robot | mean score: 0.20 ± 0.09 -> 1.00 ± 0.00 | Improved |
| compare models to human baseliners on Screening Evasion | Gemini 3.1 Pro Preview · ABC-Bench Screening Evasion | mean score: 0.22 ± 0.07 -> 0.78 ± 0.02 | Improved |
| newer Claude Opus 4.6 versus Claude Opus 4 | Claude Opus · ABC-Bench Fragment Design | mean score: 0.71 ± 0.05 -> 1.00 ± 0.00 | Improved |
| switch from creative Screening Evasion to well-documented Fragment Design | Qwen3.5 397B-A17B · ABC-Bench | mean score: 0.57 ± 0.02 -> 0.76 ± 0.02 | Improved |
| Screening Evasion prompt concealing malicious intent | Claude Sonnet 4.6, Opus 4.6, GPT-5.4 · ABC-Bench Screening Evasion | refusal rate: 1.00 | Hurt |
| model refuses Fragment Design samples | Claude Opus 4 · ABC-Bench Fragment Design | refusal rate: 0.52 | Mixed |
| LLM-guided real wet-lab Gibson assembly on OpenTrons Flex | GPT-o4-mini-high · Wet-lab DNA assembly | successful assemblies: 3 of 3 | Improved |
Closest papers by meaning
- BioAgent Bench: An AI Agent Evaluation Suite for Bioinformatics (ICML 2026 · similarity 0.90)
- Measuring Scientific Capabilities of Language Models with a Systems Biology Dry Lab (NeurIPS 2025 · similarity 0.87)
- BioXArena: Benchmarking LLM Agents on Multi-Modal Biomedical Machine Learning Tasks (NeurIPS 2026 · similarity 0.87)
- ScienceBoard: Evaluating Multimodal Autonomous Agents in Realistic Scientific Workflows (ICLR 2026 · similarity 0.86)
- AstaBench: Rigorous Benchmarking of AI Agents with a Scientific Research Suite (ICLR 2026 · similarity 0.86)
- SciAgentGym: Benchmarking Multi-Step Scientific Tool-Use in LLM Agents (ICML 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
