Research map / Large language models / Language, safety and interpretability

Towards scientific discovery with dictionary learning: Extracting biological concepts from microscopy foundation models

ICML 2025 · Large language models › Language, safety and interpretability · approach: Mechanistic analysis of attention and representations · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

The paper text provided is a draft about scaling masked autoencoder foundation models (Phenom) for microscopy phenomics, not the dictionary learning paper named in the title; it introduces perturbation consistency and Phenoprints-16M.

Problem

Evaluating and scaling vision transformer masked autoencoders trained on cell microscopy images for biological representation learning, where existing recall-based benchmarks are limited and downstream utility is unclear.

Approach

Curate Phenoprints-16M, a subset of microscopy crops with statistically significant perturbation phenotypes, to pretrain larger MAEs, and add a self-contained perturbation consistency benchmark alongside biological relationship recall.

Key results

• Scaling the MAE to Phenom-G/8 (1.9B parameters) continues to improve biological relationship recall and perturbation consistency, per the draft's claims.
• Training on the curated Phenoprints-16M improves biological relationship recall and perturbation consistency relative to the RPI-93M-trained MAE-L/8+.
• Intermediate-layer representations sometimes outperform the final layer on downstream probing tasks.

Measured effects reported in the paper

Surprising findings

• The largest DINOv2 (ViT-giant, 1.1B) performed worse on downstream benchmarks than the 25M-parameter ViT-S.

Limitations

• Text is an unfinished draft with many TODOs and placeholder citations; the dictionary learning work described in the title is absent.
• Results and numbers are not reported in the provided text.

Experimental setup

Datasets: Phenoprints-16M RPI-93M RxRx3 RxRx1 CORUM Signor Reactome PPI StringDB

Benchmarks: RxRx1 1139-class classification Phenoprint 5-class classification Anax 40-class gene group classification perturbation consistency biological relationship recall

Metrics: linear probing accuracy KS test based consistency

Baselines: DINOv2-S/14 MAE-L/8+ on RPI-93M CA-MAE-S/16

Builds on: masked autoencoder DINOv2 MIM-refiner

Closest papers by meaning

• A Cross Modal Knowledge Distillation & Data Augmentation Recipe for Improving Transcriptomics Representations through Morphological Features (ICML 2025 · similarity 0.84)
• Micro-Bench: A Microscopy Benchmark for Vision-Language Understanding (NeurIPS 2024 · similarity 0.84)
• Learning Multi-Level Features with Matryoshka Sparse Autoencoders (ICML 2025 · similarity 0.83)
• From Mechanistic Interpretability to Mechanistic Biology: Training, Evaluating, and Interpreting Sparse Autoencoders on Protein Language Models (ICML 2025 · similarity 0.83)
• Archetypal SAE: Adaptive and Stable Dictionary Learning for Concept Extraction in Large Vision Models (ICML 2025 · similarity 0.83)
• Interpreting CLIP with Sparse Linear Concept Embeddings (SpLiCE) (NeurIPS 2024 · similarity 0.83)