Research map / Large language models / Retrieval and knowledge
sciLaMA: A Single-Cell Representation Learning Framework to Leverage Prior Knowledge from Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces sciLaMA, a paired-VAE framework that adapts static gene embeddings from LLMs and protein/single-cell models to scRNA-seq data for cell representation, imputation, marker and trajectory analysis.
Problem
Single-cell RNA-seq analysis cannot easily incorporate external gene knowledge (text, sequences) into VAE-based models, while single-cell foundation LLMs are computationally expensive and less suited to tabular expression data.
Approach
Extend the siVAE paired cell/gene encoder-decoder so that gene encoders take precomputed static LLM/protein/single-cell gene embeddings as input, adapting them to the expression context via a stepwise training procedure and an alignment loss that links cell and gene latent dimensions.
Key results
- sciLaMA (avg.) achieved ARI 0.522 and NMI 0.745 on pancreas data, outperforming scVI-batch by 16.78% (ARI) and fine-tuned scGPT by 8.07% (ARI).
- sciLaMA reached iLISI 0.238 vs next-best 0.121 (GenePT-w), a 96.69% improvement, and runtime of 759 s versus 19,474 s for fine-tuned scGPT (~25-fold faster).
- On spatial imputation, sciLaMA (avg.) reached PCC 0.222 vs gimVI 0.224, i.e. comparable to the best baseline, while being better on JSD (0.283 vs 0.580) and RMSE (1.242 vs 1.243).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add external gene embeddings to sciLaMA instead of using self-informed embeddings (ablation) | sciLaMA · Human pancreatic datasets | ARI: 0.436 -> 0.522 | Helped |
| use sciLaMA instead of scGPT fine-tuning | cell representation learning · Human pancreatic datasets | runtime (s): 19,474 -> 759 | Helped |
| add external gene embeddings to sciLaMA instead of self-informed embeddings (ablation) | sciLaMA · Spatial transcriptomics data | PCC: 0.202 ± 0.027 -> 0.222 ± 0.027 | Helped |
| replace meaningful gene embeddings with randomized input (ablation) | sciLaMA · Spatial transcriptomics data | PCC: 0.222 ± 0.027 -> 0.051 ± 0.027 | Hurt |
| shuffle external gene embeddings to disrupt dimension alignment (ablation) | sciLaMA · Spatial transcriptomics data | PCC: 0.222 ± 0.027 -> 0.056 ± 0.036 | Hurt |
| add external gene embeddings instead of self-informed embeddings (ablation) | sciLaMA · Marker gene clustering | Davies-Bouldin Index: 0.977 -> 0.727 | Helped |
| vary the alignment-loss weight γ from 0 to 0.05 (ablation) | sciLaMA · Clustering performance | ARI mean: 0.464 -> 0.665 | Helped |
| vary latent dimension K from 10 to 40 (ablation) | sciLaMA · Clustering performance | ARI mean: 0.651 -> 0.680 | Helped |
| use external gene embeddings rather than self-informed embeddings (ablation) | sciLaMA · Human pancreatic datasets | batchASW: 0.832 -> 0.865 | Helped |
| use sciLaMA rather than scVI | cell representation learning · P0 mouse neurodevelopment | ARI: 0.284 -> 0.316 | Helped |
Surprising findings
- The self-informed variant sciLaMA (s.i.), which uses no external prior knowledge, still outperforms scVI on several metrics, which the authors attribute to regularization.
- In spatial imputation, sciLaMA's PCC (0.222 avg) is slightly below gimVI (0.224), so it is not uniformly best on every metric.
Limitations
- Gene embeddings are static and precomputed, so the LLM is not updated and context enters only through the adapter.
- Gains over baselines are sometimes marginal (e.g., imputation PCC is below gimVI) and rely on variant selection across many gene-embedding sources.
- Evaluation is limited to a few small datasets (pancreas, PBMC 3K, P0 cortex, spatial), and the organoid application is described but not evaluated.
Experimental setup
Datasets: Tran et al. 2020 pancreas benchmark (Baron, Segerstolpe, Muraro, Xin, Wang) Zeisel et al. 2015 Codeluppi et al. 2018 spatial transcriptomics PBMC 3K SNARE-seq P0 mouse cortex (Chen et al. 2019)
Benchmarks: Tran et al. 2020 pancreas batch-correction benchmark Codeluppi spatial transcriptomics imputation PBMC 3K marker identification P0 mouse cortex trajectory
Metrics: ARI NMI ASW batchASW iLISI cLISI PCC SCC JSD RMSE
Baselines: scVI scGPT fine-tuned scGPT zero-shot CellPLM zero-shot GenePT-w scProjection gimVI uniPort Tangram siVAE
Builds on: siVAE scVI scGPT GenePT SATURN
Closest papers by meaning
- CellVerse: Do Large Language Models Really Understand Cell Biology? (NeurIPS 2025 · similarity 0.88)
- CellPLM: Pre-training of Cell Language Model Beyond Single Cells (ICLR 2024 · similarity 0.87)
- Cell2Sentence: Teaching Large Language Models the Language of Biology (ICML 2024 · similarity 0.85)
- A Simple and Comprehensive Benchmark for Single-Cell Transcriptomics (AAAI 2025 · similarity 0.85)
- Unsupervised Gene-Cell Collective Representation Learning with Optimal Transport (AAAI 2024 · similarity 0.85)
- scPilot: Large Language Model Reasoning Toward Automated Single-Cell Analysis and Discovery (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.