Research map / Label-efficient and robust learning / Self-supervised and contrastive
Cell ontology guided transcriptome foundation model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes scCello, a transcriptome foundation model pre-trained on 22M CellxGene cells that adds cell-type coherence and cell-ontology relational alignment losses (PPR-derived similarities) to masked gene prediction, improving cell type identification, novel cell type classification, marker gene prediction, and drug response prediction.
Problem
Existing transcriptome foundation models treat cells as independent samples and ignore taxonomic relationships between cell types encoded in cell ontology graphs.
Approach
Incorporate cell ontology structure into TFM pre-training via a supervised contrastive intra-cell type coherence loss plus an inter-cellular relational alignment loss, where cell similarities in representation space are aligned with Personalized PageRank-derived, discretized ontology similarities, alongside masked gene prediction.
Key results
- Zero-shot cell type clustering: scCello reaches 0.670 AvgBio on the ID dataset versus 0.577 for scTab and 0.489 for UCE, and 0.687 OOD average.
- Novel cell type classification achieves up to 76.8% Acc for 9 novel types and 33.5% Acc for 87 novel types.
- Marker gene prediction average AUROC of 0.743 vs 0.730 for the second-best MGP+Sup; fine-tuned scCello reaches 0.867 Acc and 0.511 Macro F1 on the ID dataset.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use cell ontology-guided pre-training | scCello · ID cell type clustering | AvgBio: 0.577 -> 0.670 | Helped |
| Use cell ontology-guided pre-training | scCello · OOD cell type clustering average | AvgBio: 0.629 -> 0.687 | Helped |
| Use cell ontology-guided pre-training | scCello · ID cell type identification | Accuracy: 0.820 -> 0.867 | Helped |
| Use cell ontology-guided pre-training | scCello · Marker gene prediction average | AUROC: 0.730 -> 0.743 | Helped |
| Use scCello representations in the cancer drug response pipeline | DeepCDR pipeline · Cancer drug response prediction | PCC: 0.922 -> 0.917 | Hurt |
| Add masked gene prediction loss (ablation) | scCello · Novel cell type classification Dct1 | Accuracy: 0.287 -> 0.335 | Helped |
| Add inter-cellular ontology relation loss (ablation) | scCello · Novel cell type classification Dct1 | Accuracy: 0.147 -> 0.335 | Helped |
| Add intra-cellular cell-type coherence loss (ablation) | scCello · Novel cell type classification Dct1 | Accuracy: 0.280 -> 0.335 | Helped |
| Add cell-type representation regularization loss (ablation) | scCello · Novel cell type classification Dct1 | Accuracy: 0.296 -> 0.335 | Helped |
Surprising findings
- Removing the inter-cellular ontology relational loss reduced novel cell type classification accuracy by 56.1%, the largest ablation effect.
- Geneformer and scGPT barely outperform non-TFM baselines on zero-shot cell type clustering.
Limitations
- Cell ontology is constantly revised, requiring retraining to incorporate updates.
- In-silico knockout marker detection also flags housekeeping genes and master transcription factors, not only marker genes.
- Pre-training requires substantial compute with associated carbon emissions.
Experimental setup
Datasets: CellxGene census 2023-7-25 (22M cells) GSE96583 GSE130148 CellMarker2 PanglaoDB DeepCDR cancer drug response data
Benchmarks: curated ID and OOD cell type clustering datasets novel cell type classification (OOD cell type datasets) marker gene prediction (GSE96583, GSE130148) DeepCDR cancer drug response IC50 prediction batch integration
Metrics: NMI ARI ASW AvgBio Acc Macro F1 AUROC PCC AvgBatch Overall
Baselines: Geneformer scGPT scTab UCE scFoundation Seurat Harmony scVI scANVI DeepCDR DET
Builds on: BERT scGPT Geneformer supervised contrastive loss Personalized PageRank
Closest papers by meaning
- CellVerse: Do Large Language Models Really Understand Cell Biology? (NeurIPS 2025 · similarity 0.87)
- Semi-supervised Knowledge Transfer Across Multi-omic Single-cell Data (NeurIPS 2024 · similarity 0.87)
- Cell2Sentence: Teaching Large Language Models the Language of Biology (ICML 2024 · similarity 0.87)
- LangCell: Language-Cell Pre-training for Cell Identity Understanding (ICML 2024 · similarity 0.86)
- CellPLM: Pre-training of Cell Language Model Beyond Single Cells (ICLR 2024 · similarity 0.86)
- scPilot: Large Language Model Reasoning Toward Automated Single-Cell Analysis and Discovery (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.