Research map / Label-efficient and robust learning / Self-supervised and contrastive
Masked Autoencoders for Microscopy are Scalable Learners of Cellular Biology
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Scales masked autoencoders (U-Net and ViT, with a Fourier loss and a channel-agnostic variant) on millions of Cell Painting microscopy images, showing better recall of known biological relationships than weakly supervised and ImageNet models.
Problem
Featurizing large-scale high content screening microscopy images for biological research, where weakly supervised perturbation classifiers are sensitive to perturbation strength and fail to scale to large datasets.
Approach
Trains MAEs (U-Net and ViT) on progressively larger HCS datasets, adds a Fourier-domain L1 reconstruction loss to stabilize large ViT training, and introduces a channel-agnostic MAE (CA-MAE) that embeds images with differing channel sets at test time.
Key results
- Best MAE, ViT-L/8+ trained on RPI-93M, achieves an 11.5% relative improvement over the best WSL model (ViT-L/16 on RxRx1-2M) in hu.MAP recall.
- Recall of known relationships correlates strongly with training FLOPs for MAEs, while naive WSL scaling degrades.
- CA-MAE ViT-L/16+ trained on six-channel RPI-93M reaches 0.95 perturbation retrieval on JUMP-CP, versus 0.88 for ImageNet ViT-L/16 and 0.53 for CellProfiler.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MAE ViT-L/8+ trained on RPI-93M instead of the strongest WSL model, ViT-L/16 trained on RxRx1-2M | RxRx3 CRISPR knockout relationship retrieval · RxRx3 | hu.MAP recall: .397 -> .443 | Helped |
| Add Fourier-domain reconstruction loss (α=0.01) (ablation) | MAE ViT-L/16 trained on RPI-52M · RxRx3 CRISPR knockouts | CORUM recall: .607 -> .626 | Helped |
| Add Fourier-domain reconstruction loss (α=0.01) (ablation) | MAE ViT-L/16 trained on RPI-52M · RxRx3 CRISPR knockouts | hu.MAP recall: .414 -> .425 | Helped |
| Add Fourier-domain reconstruction loss (α=0.01) (ablation) | MAE ViT-L/16 trained on RPI-52M · RxRx3 CRISPR knockouts | StringDB recall: .460 -> .468 | Helped |
| Use channel-agnostic architecture instead of standard MAE ViT-B/16 (ablation) | MAE ViT-B/16 trained on RPI-52M · RxRx3 CRISPR knockouts | CORUM recall: .540 -> .587 | Helped |
| Use channel-agnostic architecture instead of standard MAE ViT-B/16 (ablation) | MAE ViT-B/16 trained on RPI-52M · RxRx3 CRISPR knockouts | hu.MAP recall: .373 -> .404 | Helped |
| Add Fourier-domain loss to CA-MAE ViT-B/16 (ablation) | CA-MAE ViT-B/16 trained on RPI-52M · RxRx3 CRISPR knockouts | CORUM recall: .587 -> .586 | No clear effect |
| Add Fourier-domain loss to CA-MAE ViT-B/16 (ablation) | CA-MAE ViT-B/16 trained on RPI-52M · RxRx3 CRISPR knockouts | hu.MAP recall: .404 -> .398 | Hurt |
| Apply TVN instead of no embedding transformation (ablation) | RPI-93M MAE ViT-L/8+ · RxRx3 CRISPR knockouts | CORUM recall: .124 -> .622 | Helped |
| Train MAE ViT-L/8+ through epoch 46 instead of epoch 1 (ablation) | MAE ViT-L/8+ trained on RPI-52M · RxRx3 CRISPR knockouts | hu.MAP recall: .357 -> .424 | Helped |
| Scale MAE ViT-L/8+ training from RPI-52M to RPI-93M (ablation) | MAE ViT-L/8+ · RxRx3 CRISPR knockouts | hu.MAP recall: .424 -> .443 | Helped |
| Use channel-agnostic MAE ViT-L/16+ rather than MAE ViT-L/8+ (ablation) | MAE pretrained on RPI-93M · JUMP-CP perturbation retrieval | fraction retrieved: .78 ±.13 -> .95 ±.05 | Helped |
Surprising findings
- Pretrained ImageNet ViT classifiers outperform most WSL models trained on HCS datasets in relationship recall.
- The ViT-L/16 WSL model trained on curated RxRx1-2M is the only WSL model that beats ImageNet pretraining, suggesting WSL requires carefully curated classes.
Limitations
- CA-MAE was not scaled to the best ViT-L/8+ architecture because of large token counts (6,144 for 6-channel images).
- MAE embeddings underperform CellProfiler on JUMP-CP sibling retrieval, indicating difficulty distinguishing related perturbations.
- Direct comparison with external platform (CP-DiNO) is not possible due to differing assay technology and methodology.
Experimental setup
Datasets: RxRx1 RxRx1-2M RxRx3 RPI-52M RPI-93M ImageNet-21k CORUM hu.MAP Reactome StringDB JUMP-CP (CPJUMP1)
Benchmarks: CORUM hu.MAP Reactome StringDB JUMP-CP perturbation and sibling retrieval CellProfiler feature regression
Metrics: recall of known relationships at top and bottom 5% cosine si fraction of perturbations retrieved (FDR 5%) coefficient of determination R2 recall at 5% FPR
Baselines: ImageNet-21k pretrained ViT DenseNet-161 WSL (RxRx1) ViT-L/16 WSL (RxRx1-2M) CellProfiler DiNO ViT-S/8 (CP-DiNO 1640) Pixel intensity statistics
Builds on: masked autoencoders DenseNet Typical Variation Normalization Lion optimizer multimodal MAE
Closest papers by meaning
- ViTally Consistent: Scaling Biological Representation Learning for Cell Microscopy (ICML 2025 · similarity 0.88)
- Structure-Guided Masked Autoencoders for Ultra-High Resolution Scientific Image Understanding (NeurIPS 2026 · similarity 0.87)
- Self-Guided Masked Autoencoders for Domain-Agnostic Self-Supervised Learning (ICLR 2024 · similarity 0.87)
- ChAda-ViT : Channel Adaptive Attention for Joint Representation Learning of Heterogeneous Microscopy Images (CVPR 2024 · similarity 0.87)
- Micro-Bench: A Microscopy Benchmark for Vision-Language Understanding (NeurIPS 2024 · similarity 0.85)
- A Cross Modal Knowledge Distillation & Data Augmentation Recipe for Improving Transcriptomics Representations through Morphological Features (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.