AI Paper Atlas

Research map / Learning theory and algorithms / Prediction and decision losses

A low-rank decoder bottleneck bounds reliability in foundation-model perturbation prediction

Zheyu Zhang, Yuanhao Huang, Fan Feng, Jie Liu

NeurIPS 2026 · Learning theory and algorithms › Prediction and decision losses · approach: Probabilistic calibration · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Audits the state-transition decoder used by foundation-model perturbation-response predictors and finds a rank-1 attention bottleneck that limits per-prediction reliability.

Problem

Foundation-model predictors of single-cell perturbation response are judged mainly by aggregate accuracy and lack principled per-prediction reliability criteria.

Approach

Audits the shared state-transition (ST) decoder across three encoders and four cell lines under CRISPRi perturbations, identifying a rank-1 attention bottleneck whose leading direction maps in gene space, which is argued to bound prediction reliability.

Key results

Measured effects reported in the paper

What was changedWhereMeasuredEffect
project L0 attention output to rank 1 (ablation)SE+ST · Replogle-Nadig CRISPRi held-out (mean over 4 cell lines)mean PDC: 0.374 -> 0.372No clear effect
project L0 attention output to rank 1 (ablation)scGPT+ST · Replogle-Nadig CRISPRi held-out (mean over 4 cell lines)mean PDC: 0.337 -> 0.265Hurt
replace L0 attention output with a random direction (ablation)SE+ST · CRISPRi held-outPDCHurt
replace output of L1 to L7 with random direction (ablation)ST decoder · CRISPRi held-outPDCNo clear effect
set top L0 component size to mean over perturbations (ablation)SE+ST and Tahoe+ST · CRISPRi held-outPDCNo clear effect
swap L0 direction with another perturbation's direction (ablation)ST decoder · CRISPRi held-outPDCHurt
substitute another encoder's direction at L0 (ablation)ST decoder · CRISPRi held-outPDC: -0.1 to -0.3Hurt
project L0 output onto 32 shared training-estimated directions vs random directions (ablation)ST decoder · CRISPRi held-outPDCImproved
replace decoder with ridge regression on its inputsencoder+ST · CRISPRi held-outmean PDC: within 0.04 PDCNo clear effect
evaluate on perturbations far from trainingencoder+ST decoders · Replogle-Nadig CRISPRi, far from training (cos dist > 0.5)mean PDC: 0.07-0.14 vs 0.07 train-meanHurt
oracle nearest-training-response averaging instead of decoderoracle retrieval · CRISPRi held-outPDC: +0.137 PDC on averageImproved
use predicted-NN-distance as confidence scoreST decoders · Replogle-Nadig CRISPRi held-out (mean over 4 cell lines)Spearman |rho| with PDC: 0.88 -> 0.81Improved

Experimental setup

Datasets: CRISPRi perturbation single-cell data (four cell lines)

Builds on: state-transition model (Adduri et al. 2025)

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy