Research map / Architectures and efficiency / RNNs and normalization
ViTally Consistent: Scaling Biological Representation Learning for Cell Microscopy
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Curates a self-consistent 16M-image microscopy dataset (Phenoprints-16M), trains a 1.9B-parameter ViT-G/8 MAE on it, and selects the best intermediate block via linear probes to improve cell-microscopy representations.
Problem
Learning representations of cell microscopy images that capture biologically meaningful perturbation effects is limited by experimental data availability and by default use of the final encoder layer, which may not be optimal.
Approach
Curate a perturbation-consistent, class-balanced pretraining set using weaker models and a permutation-based consistency test, scale an MAE to ViT-G/8, and pick the optimal intermediate ViT block using cheap linear probes (RxRx1 siRNA, Anax gene groups) that correlate with whole-genome benchmarks.
Key results
- MAE-G/8 trimmed to block 38 achieves 0.63 KS and 18.2 CM replicate consistency vs 0.52 and 12.3 for MAE-L/8 RPI-93M, and the highest whole-genome recall (45.4%).
- On RxRx1 linear probing, the block-38 features of MAE-G/8 reach balanced accuracy 0.51, about 60% higher than the final block of MAE-L/8+.
- On RxRx3-core compound-gene activity, trimmed MAE-G/8 reaches average precision 0.309 (z=5.38), a 42% relative improvement over MAE-L/8 RPI-93M (3.77 z).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train MAE-L/8 on curated Phenoprints-16M instead of RPI-93M | MAE-L/8 · RxRx3 whole genome | KS: .52 -> .59 | Helped |
| Train MAE-L/8 on curated Phenoprints-16M instead of RPI-93M | MAE-L/8 · RxRx3 whole genome | biological relationship recall: 44.4 { $\pm$ .12} -> 44.4 { $\pm$ .12} | No clear effect |
| Scale from MAE-L/8 to MAE-G/8 while training on Phenoprints-16M | MAE trained on Phenoprints-16M · RxRx3 whole genome | KS: .59 -> .60 | Helped |
| Scale from MAE-L/8 to MAE-G/8 while training on Phenoprints-16M | MAE trained on Phenoprints-16M · RxRx3 whole genome | biological relationship recall: 44.4 { $\pm$ .12} -> 45.4 { $\pm$ .07} | Helped |
| Use the selected earlier block instead of the final block (ablation) | MAE-G/8 PP-16M · RxRx3-core compound-gene benchmark | average precision: 0.302 { $\pm$ .015} -> 0.309 { $\pm$ .015} | Helped |
| Use the selected earlier block instead of the final block (ablation) | MAE-L/8 RPI-93M · RxRx3-core compound-gene benchmark | average precision: 0.290 { $\pm$ .017} -> 0.299 { $\pm$ .016} | Helped |
| Use block 38 instead of the final block (ablation) | MAE-G/8 PP-16M · RxRx3 whole genome | KS: .60 -> .63 | Helped |
| Use block 38 instead of the final block (ablation) | MAE-G/8 PP-16M · RxRx3 whole genome | CM: 16.4 -> 18.2 | Helped |
| Use block 38 instead of the final block (ablation) | MAE-G/8 PP-16M · RxRx3 whole genome | biological relationship recall: 45.4 { $\pm$ .07} -> 45.4 { $\pm$ .15} | No clear effect |
| Use intermediate block 38 instead of the final block (ablation) | MAE-G/8 PP-16M · RxRx1 siRNA genetic perturbation classification | balanced accuracy: 8.5% | Helped |
| Use intermediate block instead of the final block (ablation) | MAE-G/8 PP-16M · Anax 40-class functional gene group classification | balanced accuracy: 5% | Helped |
| Train a Dino-V2 ViT-L/16 on RxRx3 instead of CA-MAE-S/16 on RxRx3 | RxRx3 microscopy-pretrained encoder · RxRx3-core compound-gene benchmark | average precision: 0.273 { $\pm$ .016} -> 0.258 $\pm$ .015 | Hurt |
Surprising findings
- Intermediate blocks outperform the final block for MAEs, DINOv2 and natural-image models, with the final block sometimes performing worse than an untrained ViT-S.
- Dino-V2 pretraining on microscopy data overfit and failed to beat the CA-MAE baseline on RxRx3-core, so the authors used MAEs instead.
Limitations
- Dino-V2 recipe for microscopy data could not be made effective, so SSL comparisons are limited to MAEs.
- Recall on JUMP-CP is considerably lower than on RxRx3, attributed to different assay protocols and variance.
- Fourier loss implementation differed from paper description; loss-design ablation left to future work.
Experimental setup
Datasets: Phenoprints-16M RPI-93M RxRx3 RxRx1 RxRx3-core JUMP-CP Anax
Benchmarks: RxRx1 siRNA classification Anax 40-class gene group classification biological relationship recall (CORUM, hu.MAP, Reactome, StringDB) replicate consistency (KS, CM) JUMP-CP relationship recall RxRx3-core compound-gene activity
Metrics: balanced accuracy recall % KS statistic Cramer-von Mises statistic average precision z-score
Baselines: MAE-L/8 RPI-93M CA-MAE-S/16 RxRx3 DINOv2 ViT-S/L/G ViT-L/16 ImageNet MAE CellProfiler Untrained ViT-S/16
Builds on: MAE Typical Variation Normalization DINOv2 Channel-agnostic MAE
Closest papers by meaning
- Masked Autoencoders for Microscopy are Scalable Learners of Cellular Biology (CVPR 2024 · similarity 0.88)
- Micro-Bench: A Microscopy Benchmark for Vision-Language Understanding (NeurIPS 2024 · similarity 0.87)
- Towards Scalable Context-Aware Single-Cell Spatial Transcriptomics Prediction from Histology Images (NeurIPS 2026 · similarity 0.87)
- MorphoHELM: A Comprehensive Benchmark for Evaluating Representations for Microscopy-Based Morphology Assays (NeurIPS 2026 · similarity 0.87)
- CellCLIP - Learning Perturbation Effects in Cell Painting via Text-Guided Contrastive Learning (NeurIPS 2025 · similarity 0.87)
- A Cross Modal Knowledge Distillation & Data Augmentation Recipe for Improving Transcriptomics Representations through Morphological Features (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.