Research map / Vision-language and multimodal / CLIP and zero-shot
LangCell: Language-Cell Pre-training for Cell Identity Understanding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LangCell, a language-cell pre-training framework that aligns Geneformer-initialized cell encoder embeddings with PubMedBERT-based text descriptions (scLibrary, 27.5M cell-text pairs), enabling zero-shot cell identity understanding.
Problem
Single-cell pre-trained language models learn only from transcriptomics and need labeled fine-tuning data to perform cell identity understanding (cell type, disease, pathway), which fails when labels are scarce or for novel cell types.
Approach
Jointly pre-train a cell encoder and a cell-text dual/cross-attention text encoder with masked gene modeling, cell-cell contrastive, cell-text contrastive, and cell-text matching losses, using OBO Foundry-enriched multi-aspect text descriptions so that cell identity can be queried zero-shot via text.
Key results
- Zero-shot LangCell reaches 86.5% accuracy and 89.6% macro F1 on PBMC10K, where all single-cell PLM baselines cannot perform zero-shot and few-shot baselines reach at most 88.0% accuracy at 9-shot (Geneformer).
- On NSCLC subtype classification, zero-shot LangCell gets 93.5% accuracy and 93.2% macro F1, versus 73.1% for Geneformer with 10-shot fine-tuning.
- LangCell-CE achieves 98.3% accuracy on PBMC10K fine-tuning and 85.4% on Zheng68K, outperforming Geneformer (97.8% and 83.9%) and scGPT.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| pre-train with cell-text alignment and matching (ablation) | LangCell · PBMC10K zero-shot cell type annotation | accuracy: 85.8 -> 86.5 | Helped |
| pre-train with cell-text alignment and matching (ablation) | LangCell · PBMC3&68K zero-shot cell type annotation | accuracy: 83.7 -> 83.9 | Helped |
| add CTM matching module (ablation) | LangCell · PBMC10K zero-shot cell type annotation | macro F1: 89.4 -> 89.6 | Helped |
| add cell-cell contrastive learning (C-C) to MGM (ablation) | Geneformer-initialized cell encoder · fine-tune cell type annotation average | accuracy: 76.1 -> 77.0 | Helped |
| add text-based pre-training tasks to the first-stage model (ablation) | LangCell-1 · fine-tune cell type annotation average | accuracy: 77.0 -> 78.0 | Helped |
| combine cell-text similarity and cell-text matching scores instead of using matching alone (ablation) | LangCell · PBMC10K zero-shot cell type annotation | accuracy: 56.83 -> 86.54 | Helped |
| vary alpha to balance similarity and matching scores (ablation) | LangCell · zero-shot cell type annotation on PBMC10K and PBMC3&68K | accuracy: 56.83; 92.40; 86.54; 65.32; 88.54; 83.94 | Mixed |
| increase k, the number of candidates scored by the matching module (ablation) | LangCell · PBMC10K zero-shot cell type annotation | accuracy: 86.76; 86.28; 86.04; 86.53; 86.54 | Mixed |
| use two-stage pre-training, with MGM and C-C before multimodal training (ablation) | LangCell · pre-training convergence | convergence speed | Helped |
| initialize cell and text encoders from Geneformer and PubMedBERT rather than training from scratch (ablation) | LangCell · pre-training convergence | loss decline: loss demonstrates negligible decline throughout the first 20,000 steps | Helped |
| introduce dropout-zero noise in bottom-30%-expressed genes (ablation) | LangCell · PBMC10K zero-shot cell type annotation | accuracy: 86.54 -> 86.66; 86.37; 88.14 | No clear effect |
Surprising findings
- Without the cell-text matching module (k=0), LangCell's zero-shot accuracy is nearly unchanged (84.8% vs 85.2% average), so the costly CTM re-ranking adds little beyond cosine similarity.
- Zero-shot performance on Tabula Sapiens cell types not covered by scLibrary (26.74 Recall@1) is close to performance on covered types (28.77), suggesting transfer to unseen cell types.
Limitations
- Pre-training texts come only from OBO Foundry ontologies, limiting textual diversity.
- Cannot analyze other single-cell omics such as scATAC-seq and does not support cell or text generation.
- Pre-training took about 50 days on four A100 GPUs, and training from scratch was infeasible in the authors' experiments.
Experimental setup
Datasets: scLibrary PBMC10K PBMC3&68K Zheng68K LiverCross (macParland/aizarani) Perirhinal Cortex Tabula Sapiens
Benchmarks: PBMC10K PBMC3&68K Zheng68K LiverCross Perirhinal Cortex Tabula Sapiens NSCLC subtype classification (constructed) MSigDB hallmark pathway identification (constructed)
Metrics: accuracy macro F1 recall@k AUROC AUPRC Avg_bio ASW_batch S_final
Baselines: scBERT scGPT Geneformer xTrimoGene BioTranslator scVI
Builds on: Geneformer PubMedBERT BioTranslator ALBEF
Closest papers by meaning
- CellPLM: Pre-training of Cell Language Model Beyond Single Cells (ICLR 2024 · similarity 0.89)
- CellVerse: Do Large Language Models Really Understand Cell Biology? (NeurIPS 2025 · similarity 0.87)
- Cell2Sentence: Teaching Large Language Models the Language of Biology (ICML 2024 · similarity 0.87)
- Cell ontology guided transcriptome foundation model (NeurIPS 2024 · similarity 0.86)
- scPilot: Large Language Model Reasoning Toward Automated Single-Cell Analysis and Discovery (NeurIPS 2025 · similarity 0.85)
- scSSL-Bench: Benchmarking Self-Supervised Learning for Single-Cell Data (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.