Research map / Image generation / Autoregressive and latent generation
HEST-1k: A Dataset For Spatial Transcriptomics and Histology Image Analysis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces HEST-1k, a curated collection of 1,229 paired spatial transcriptomics and H&E whole-slide image samples with the HEST-Library toolkit and HEST-Benchmark of nine gene-expression-from-histology tasks used to evaluate 11 pathology foundation models.
Problem
Spatial transcriptomics data is scarce, heterogeneous and unstandardized, and paired ST-histology resources are lacking for large-scale computational pathology and for benchmarking pathology foundation models beyond saturated diagnostic tasks.
Approach
Aggregate and re-align public and internal ST and H&E cohorts into a unified format (generic TIFF, AnnData, re-estimated resolution, automatic Visium fiducial alignment, CellViT nuclear segmentation), and define patch-level gene expression prediction tasks as a benchmark for foundation models.
Key results
- H-Optimus-0 achieves the best HEST-Benchmark average Pearson correlation (0.4146 with PCA+Ridge), slightly ahead of UNIv1.5 (0.4090).
- Model size correlates with HEST-Benchmark average performance (Pearson R=0.81, P<0.01) following a logarithmic trend.
- CONCH-FT (fine-tuned on five Xenium breast samples) outperforms CONCH on most BCNB ER/PR/HER2 metrics, e.g., ER AUC 0.884 vs 0.881.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fine-tune CONCH on five Xenium breast cancer cases using multimodal contrastive alignment | CONCH · BCNB ER status | AUC: 0.881 -> 0.884 | Helped |
| Fine-tune CONCH on five Xenium breast cancer cases using multimodal contrastive alignment | CONCH · BCNB ER status | Balanced accuracy: 0.745 -> 0.752 | Helped |
| Fine-tune CONCH on five Xenium breast cancer cases using multimodal contrastive alignment | CONCH · BCNB PR status | AUC: 0.810 -> 0.818 | Helped |
| Fine-tune CONCH on five Xenium breast cancer cases using multimodal contrastive alignment | CONCH · BCNB PR status | Balanced accuracy: 0.698 -> 0.714 | Helped |
| Fine-tune CONCH on five Xenium breast cancer cases using multimodal contrastive alignment | CONCH · BCNB HER2 status | AUC: 0.715 -> 0.724 | Helped |
| Fine-tune CONCH on five Xenium breast cancer cases using multimodal contrastive alignment | CONCH · BCNB HER2 status | Balanced accuracy: 0.624 -> 0.615 | Hurt |
| Increase the number of trainable vision-encoder parameters | HEST-Benchmark foundation models · HEST-Benchmark average | Pearson correlation: R=0.81, P-value<0.01 | Helped |
| Increase the number of pretraining image patches | HEST-Benchmark foundation models · HEST-Benchmark average | Pearson correlation: R=0.48, P-value=0.13 | No clear effect |
Surprising findings
- ResNet50 pretrained on ImageNet is outperformed by all histology-specific encoders, but the gap between top and bottom domain-specific models is only moderate, and the data scaling correlation with performance is weak (R=0.48, P=0.13).
- Performance remains low on some tasks (e.g., READ ~0.23 Pearson), suggesting morphology may poorly reflect expression for certain cancers.
Limitations
- Spatial transcriptomics data is inherently noisy, with staining, compression and batch-effect artifacts that may degrade label quality.
- HEST-Library cannot cover all existing data formats and is a blueprint rather than a complete solution.
- The CONCH fine-tuning result is based on only five paired WSIs, limiting conclusions.
Experimental setup
Datasets: HEST-1k BCNB TCGA Camelyon16 PanNuke
Benchmarks: HEST-Benchmark BCNB
Metrics: Pearson correlation ROC-AUC balanced accuracy
Baselines: ResNet50 (IN) CTransPath Phikon CONCH REMEDIS GigaPath UNI Virchow Virchow2 UNIv1.5 H-Optimus-0
Builds on: CONCH CellViT YOLOv8 VALIS Scanpy
Closest papers by meaning
- Diffusion Generative Modeling for Spatially Resolved Gene Expression Inference from Histology Images (ICLR 2025 · similarity 0.92)
- STimage-1K4M: A histopathology image-gene expression dataset for spatial transcriptomics (NeurIPS 2024 · similarity 0.91)
- Scalable Generation of Spatial Transcriptomics from Histology Images via Whole-Slide Flow Matching (ICML 2025 · similarity 0.90)
- MEATRD: Multimodal Anomalous Tissue Region Detection Enhanced with Spatial Transcriptomics (AAAI 2025 · similarity 0.89)
- Towards Scalable Context-Aware Single-Cell Spatial Transcriptomics Prediction from Histology Images (NeurIPS 2026 · similarity 0.89)
- Preserving DEG Rankings for Gene Discovery in Histology-Based Spatial Gene Expression Prediction (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.