Research map / Large language models / Agents, code and math
Expert-level protocol translation for self-driving labs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a human-free three-stage framework that turns natural-language experimental protocols into protocol dependence graphs (syntax, semantics, execution levels) for self-driving labs, evaluated against human experts on 75 protocols.
Problem
Translating human-oriented experimental protocols into machine-executable, structured, complete, and execution-safe representations for self-driving laboratories currently requires labor-intensive manual work by domain experts and IT specialists.
Approach
Incrementally constructs a Protocol Dependence Graph: operation dependence synthesis via DSL program synthesis (EM-based joint optimization) for syntax, reagent flow analysis via reaching definitions and a PDA-style traversal for semantics, and constraint-based partial execution traces for spatial-temporal checks at the execution level, with LLMs used as auxiliary extractors.
Key results
- Translator reaches over 85% average performance relative to human experts across all indicators and significantly outperforms Best-Baseline (t(148)=-17.71, p<.0005).
- On the syntax level, Ours-SY significantly outperforms ConDec-SY (t(148)=-17.07) and DSL-LLM-SY (t(148)=-15.47).
- On the semantics level, Ours-SE significantly outperforms NL-RAG-LLM-SE (t(148)=-2.52, p<.05) and NL-LLM-SE (t(148)=-3.07, p<.005).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the three-stage protocol translation framework instead of Best-Baseline | Protocol translation · 75-protocol test set | overall translation performance | Helped |
| Use operation-dependence synthesis instead of ConDec-SY | Syntax-level translation · Protocol translation test set | syntax-level translation performance | Helped |
| Use operation-dependence synthesis instead of DSL-LLM-SY | Syntax-level translation · Protocol translation test set | syntax-level translation performance | Helped |
| Use deductive verification instead of NL-RAG-LLM-SE | Semantics-level translation · Protocol translation test set | semantics-level translation performance | Helped |
| Use deductive verification instead of NL-LLM-SE | Semantics-level translation · Protocol translation test set | semantics-level translation performance | Helped |
Surprising findings
- Translator diverges from human experts on protocols lacking a clear linear description, where experts consider the overall experimental process while the translator focuses on adjacent sentences.
Limitations
- Relies on LLM-based entity extraction and reagent kill/define judgments, with pseudo-labels that may be noisy.
- Evaluation metrics (ROUGE, BLEU, exact match on JSON) are less advanced and do not use LLM judges; the authors acknowledge this.
- Computational complexity is high for long protocols and the paper notes substantial room for efficiency improvements.
Experimental setup
Datasets: open-access experimental protocols from Protocol Exchange, Star-protocols, Bio-protocol, Current Protocols, and JoVE
Benchmarks: 75-experiment test set (1,166 steps) across chemical synthesis, genetics, medical, bioengineering, ecology
Metrics: ROUGE BLEU exact match over key-value pairs
Baselines: ConDec-SY DSL-LLM-SY NL-RAG-LLM-SE NL-LLM-SE Best-Baseline human experts
Builds on: XDL reaching definitions context-free grammar AutoDSL
Closest papers by meaning
- Hierarchically Encapsulated Representation for Protocol Design in Self-Driving Labs (ICLR 2025 · similarity 0.92)
- AI-Researcher: Autonomous Scientific Innovation (NeurIPS 2025 · similarity 0.86)
- SciResearchBench: Benchmarking AI Agents on Complex Scientific Literature Discovery (NeurIPS 2026 · similarity 0.83)
- From Reproduction to Replication: Evaluating Research Agents with Progressive Code Masking (ICLR 2026 · similarity 0.83)
- ScienceBoard: Evaluating Multimodal Autonomous Agents in Realistic Scientific Workflows (ICLR 2026 · similarity 0.83)
- LaMPilot: An Open Benchmark Dataset for Autonomous Driving with Language Model Programs (CVPR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.