Research map / Large language models / Agents, code and math
SC-Arena: A Natural Language Benchmark for Single-Cell Reasoning with Knowledge-Augmented Evaluation
Jiahao Zhao, Feng Jiang, Shaowei Qin, Zhonghui Zhang, Junhao Liu, Guibing Guo, Hamid Alinejad-Rokny, Min Yang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| scale Qwen model from 7B Qwen2.5 to 235B Qwen3 | Qwen family · SC-Arena total (5 tasks) | total score: 202.6 -> 262.7 | Improved |
| upgrade from Qwen2.5-32B to Qwen3-32B | Qwen · SC-Arena total (5 tasks) | total score: 238.06 -> 252.00 | Improved |
| scale Qwen3 from 8B to 14B | Qwen3 · SC-Arena cell generation (CG) | score: 50.53 -> 15.50 | Hurt |
| use small domain-specialized C2S-Pythia-410m instead of GPT-4o | cell type annotation · SC-Arena cell type annotation (CTA) | score: 36.29 -> 47.34 | Improved |
| use domain-specialized scGenePT instead of general Qwen3-32B | perturbation prediction · SC-Arena perturbation prediction (PP) | score: 37.54 -> 26.03 | Hurt |
| use reasoning-specialized Cell-o1 instead of Kimi-K2 | SC-Arena · SC-Arena total (5 tasks) | total score: 277.16 -> 234.20 | Hurt |
| use traditional NLP metrics instead of knowledge-augmented scoring | evaluation · cell type annotation | BERTScore/BLEU/ROUGE | Hurt |
| ground evaluator scores with cell ontology distance | knowledge-augmented evaluator · CTA | Spearman rho: rho = 0.6212 | Improved |
Closest papers by meaning
- CellVerse: Do Large Language Models Really Understand Cell Biology? (NeurIPS 2025 · similarity 0.92)
- scPilot: Large Language Model Reasoning Toward Automated Single-Cell Analysis and Discovery (NeurIPS 2025 · similarity 0.91)
- Cell2Sentence: Teaching Large Language Models the Language of Biology (ICML 2024 · similarity 0.89)
- CellPLM: Pre-training of Cell Language Model Beyond Single Cells (ICLR 2024 · similarity 0.87)
- Cell ontology guided transcriptome foundation model (NeurIPS 2024 · similarity 0.87)
- sciLaMA: A Single-Cell Representation Learning Framework to Leverage Prior Knowledge from Large Language Models (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
