Research map / Detection and segmentation / CNN features
SToFM: a Multi-scale Foundation Model for Spatial Transcriptomics
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SToFM, a multi-scale spatial transcriptomics foundation model that combines a domain-adapted Geneformer cell encoder with an SE(2) Transformer over sub-slices containing virtual cells, pretrained on the 88M-cell SToCorpus-88M.
Problem
Existing spatial transcriptomics foundation models do not capture and integrate macro-scale tissue morphology, micro-scale cell neighborhood interactions, and gene-scale expression together.
Approach
Split each ST slice into ~1000-cell sub-slices, add Leiden-clustered virtual cells for macro context, encode cells with a domain-adapted Geneformer, and model spatial relations with an SE(2)-invariant Transformer trained with masked cell modeling and pairwise distance recovery.
Key results
- Tissue region segmentation: SToFM average embryo macro F1 0.8046 vs CellPLM 0.7722; cross-slice embryo F1 0.4588 vs 0.3985 for CellPLM.
- Cell type annotation: SToFM Brain1 macro F1 0.4951 and Brain2 macro F1 0.8362, outperforming CellPLM (0.4186 and 0.7332).
- Imputation on SkinCross: Pearson 0.4877 vs 0.3985 for the next best (CellPLM).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| integrate gene, micro, and macro scales | SToFM · EmbryoCross | Macro F1: 0.4291 -> 0.4588 | Helped |
| add domain adaptation to the cell encoder (ablation) | SToFM · EmbryoCross | Macro F1: 0.3745 -> 0.4155 | Helped |
| add domain adaptation to the cell encoder (ablation) | SToFM · Brain1 | Macro F1: 0.3853 -> 0.4725 | Helped |
| add virtual cells for macro-scale information (ablation) | SToFM · EmbryoCross | Macro F1: 0.4291 -> 0.4588 | Helped |
| add virtual cells for macro-scale information (ablation) | SToFM · Brain1 | Macro F1: 0.4893 -> 0.4951 | Helped |
| add micro-scale cell interactions (ablation) | SToFM · EmbryoCross | Macro F1: 0.425 -> 0.459 | Helped |
| use the spatial distance matrix (ablation) | SToFM · EmbryoCross | Macro F1: 0.413 -> 0.459 | Helped |
| increase sub-slice scale from 10 to 1000 (ablation) | SToFM · Embryo2 | Macro F1: 0.7066 -> 0.8014 | Helped |
| increase sub-slice scale from 1000 to 2000 (ablation) | SToFM · EmbryoCross | Macro F1: 0.4588 -> 0.4554 | No clear effect |
| vary Leiden resolution from 1.0 to 0.1, 0.5, or 1.5 (ablation) | SToFM · EmbryoCross | Macro F1: 0.4588; 0.4410; 0.4572; 0.4403 | Mixed |
| increase second-time cell-encoding sample number from 0 to 12 (ablation) | SToFM · Embryo2 | Macro F1: 0.722 -> 0.758 | Helped |
| pretrain with the full dataset rather than 12.5% of SToCorpus-88M (ablation) | SToFM · Embryo2 | Macro F1: 0.758 -> 0.801 | Helped |
Surprising findings
- Full-parameter fine-tuning raises Brain1 macro F1 from 0.4951 to 0.5532, showing head tuning understates the model's potential.
Limitations
- Models only three scales (macro, micro, gene); other scales are left for future work.
- Virtual cells are embedded from averaged cell embeddings rather than gene expression, since averaged profiles are out-of-distribution.
- Benchmark evaluations use head tuning only in the main text, so comparisons reflect representation quality rather than best achievable fine-tuned performance.
Experimental setup
Datasets: SToCorpus-88M human embryo Stereo-seq (HESTA) DLPFC 10x Visium mouse brain Stereo-seq SeekSpace mouse brain mouse liver Stereo-seq human skin 10x Xenium MERFISH mouse brain
Benchmarks: human embryo structure segmentation DLPFC layer segmentation mouse brain cell type annotation mouse liver spatial deconvolution human skin imputation MERFISH zero-shot clustering
Metrics: accuracy macro F1 Pearson correlation RMSE MAE NMI ARI
Baselines: scGPT Geneformer Nicheformer CellPLM
Builds on: Geneformer Uni-Mol CellPLM Leiden clustering
Closest papers by meaning
- Diffusion Generative Modeling for Spatially Resolved Gene Expression Inference from Histology Images (ICLR 2025 · similarity 0.88)
- STINR: Deciphering Spatial Transcriptomics via Implicit Neural Representation (CVPR 2025 · similarity 0.87)
- Scalable Generation of Spatial Transcriptomics from Histology Images via Whole-Slide Flow Matching (ICML 2025 · similarity 0.87)
- Global Context-aware Representation Learning for Spatially Resolved Transcriptomics (ICML 2025 · similarity 0.87)
- STimage-1K4M: A histopathology image-gene expression dataset for spatial transcriptomics (NeurIPS 2024 · similarity 0.87)
- Towards Scalable Context-Aware Single-Cell Spatial Transcriptomics Prediction from Histology Images (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.