Research map / Large language models / Agents, code and math
scPilot: Large Language Model Reasoning Toward Automated Single-Cell Analysis and Discovery
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces scPilot, an LLM framework for omics-native reasoning that lets an LLM iteratively inspect scRNA-seq summaries and call bioinformatics tools for cell-type annotation, trajectory inference, and GRN prediction, plus the scBench benchmark of 9 datasets.
Problem
Single-cell analyses (cell-type annotation, trajectory reconstruction, TF-target prediction) rely on opaque tool pipelines or human reasoning, and LLM tool agents mostly wrap default tool outputs without biological reasoning over the data; no benchmark evaluates such language-native reasoning with g
Approach
Convert each single-cell query into a textual sketch and let an LLM iteratively propose hypotheses, invoke primitive bio-tool operators on the raw data, and revise its claims, producing an auditable reasoning trace (omics-native reasoning), without fine-tuning the LLM.
Key results
- o1 under scPilot improves average cell-type annotation accuracy by about 11% over direct prompting per the abstract (e.g., PBMC3k 0.792 vs 0.667).
- Gemini-2.5-Pro under scPilot achieves the lowest GED-nx on Pancreas (5.00) and Liver (3.33), with the abstract reporting a 30% reduction in graph-edit distance versus one-shot prompting.
- o1 under scPilot reaches GRN AUROC 0.873 on Stomach versus 0.827 for direct prompting and 0.723 for the best GNN baseline (GCN).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use iterative omics-native reasoning instead of one-shot prompting | o1 for PBMC3k cell-type annotation · PBMC3k | cluster-level accuracy: 0.667 { $\pm$ 0.005} -> 0.792 { $\pm$ 0.005} | Helped |
| Use iterative omics-native reasoning instead of one-shot prompting | o1 for Liver cell-type annotation · Liver | cluster-level accuracy: 0.560 { $\pm$ 0.001} -> 0.518 { $\pm$ 0.001} | Hurt |
| Use iterative omics-native reasoning instead of one-shot prompting | o1 for Retina cell-type annotation · Retina | cluster-level accuracy: 0.474 { $\pm$ 0.002} -> 0.728 { $\pm$ 0.007} | Helped |
| Use iterative omics-native reasoning instead of one-shot prompting | Gemini 2.5 Pro for trajectory reconstruction · Pancreas, Liver, and Neocortex | GED-nx (10s): 8.33 { $\pm$ 2.08} -> 5.00 { $\pm$ 1.73}; 8.00 { $\pm$ 4.00} -> 3.33 { $\pm$ 2.31}; 13.33 { $\pm$ 1.15} -> 9.50 { $\pm$ | Helped |
| Use iterative omics-native reasoning instead of single-step prompting | o1 for GRN prediction · Stomach, Liver, and Kidney | AUROC: 0.827 { $\pm$ 0.002} -> 0.873 { $\pm$ 0.004}; 0.753 { $\pm$ 0.001} -> 0.760 { $\pm$ 0.000}; 0.777 { $\pm$ 0.000} -> 0.79 | Helped |
| Add dataset contextual metadata (ablation) | scPilot cell-type annotation · PBMC3k | cluster-level accuracy: 0.688 -> 0.792 (o1); 0.583 -> 0.646 (GPT-4o); 0.416 -> 0.604 (GPT-4o-mini) | Helped |
| Use accurate TF-gene Gene Ontology annotations (ablation) | scPilot GRN prediction · Stomach | AUROC: 0.813 -> 0.873 (o1); 0.710 -> 0.800 (GPT-4o); 0.617 -> 0.697 (GPT-4o-mini) | Helped |
| Use accurate Monocle cluster relationships and pseudotime (ablation) | scPilot trajectory inference with o1 · Liver | GED-nx (10s): 11.33 -> 8.00 | Helped |
| Use K=10 rather than K=5 marker genes (ablation) | scPilot cell-type annotation with o1 · PBMC3k | cluster-level accuracy: 0.771 -> 0.792 | Helped |
| Use K=10 rather than K=20 marker genes (ablation) | scPilot cell-type annotation with o1 · PBMC3k | cluster-level accuracy: 0.750 -> 0.792 | Helped |
| Use K=10 rather than K=5 marker genes (ablation) | scPilot cell-type annotation with GPT-4o-mini · PBMC3k | cluster-level accuracy: 0.500 -> 0.604 | Helped |
| Use K=10 rather than K=20 marker genes (ablation) | scPilot cell-type annotation with GPT-4o-mini · PBMC3k | cluster-level accuracy: 0.542 -> 0.604 | Helped |
Surprising findings
- scPilot underperforms direct prompting in 21 of 108 comparisons, 13 of which come from mini models, and on Liver the o1 scPilot model overthinks and merges hepatocytes with hepatoblasts.
- Gemma-3-27B lags far behind proprietary models, and open-source deployment required about 15x more time than GPT-4o, so the authors used API models only.
Limitations
- Data compression into text sketches may miss signals from rare cell populations.
- Mini models frequently over-explore and hallucinate in extended reasoning chains, sometimes underperforming direct prompting.
- Results are reported on a small number of datasets and tasks; GRN benchmark uses only 46-142 questions per tissue.
Experimental setup
Datasets: PBMC3k Liver (mouse, cell-type) Retina Pancreas Liver (trajectory) Neocortex GRNdb stomach GRNdb liver GRNdb kidney TRRUST v2
Benchmarks: scBench
Metrics: cluster-level accuracy Node-Jaccard graph edit distance spectral distance AUROC
Baselines: CellTypist CellMarker 2.0 GPTCelltype Biomni py-Monocle GCN GAT GraphSAGE LLM4GRN BioGPT
Builds on: Scanpy Monocle 3 pySCENIC GPTCelltype
Closest papers by meaning
- CellVerse: Do Large Language Models Really Understand Cell Biology? (NeurIPS 2025 · similarity 0.90)
- Semi-supervised Knowledge Transfer Across Multi-omic Single-cell Data (NeurIPS 2024 · similarity 0.87)
- CellPLM: Pre-training of Cell Language Model Beyond Single Cells (ICLR 2024 · similarity 0.86)
- Cell2Sentence: Teaching Large Language Models the Language of Biology (ICML 2024 · similarity 0.86)
- A Simple and Comprehensive Benchmark for Single-Cell Transcriptomics (AAAI 2025 · similarity 0.86)
- Cell ontology guided transcriptome foundation model (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.