Research map / Learning theory and algorithms / Prediction and decision losses
A low-rank decoder bottleneck bounds reliability in foundation-model perturbation prediction
Zheyu Zhang, Yuanhao Huang, Fan Feng, Jie Liu
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Audits the state-transition decoder used by foundation-model perturbation-response predictors and finds a rank-1 attention bottleneck that limits per-prediction reliability.
Problem
Foundation-model predictors of single-cell perturbation response are judged mainly by aggregate accuracy and lack principled per-prediction reliability criteria.
Approach
Audits the shared state-transition (ST) decoder across three encoders and four cell lines under CRISPRi perturbations, identifying a rank-1 attention bottleneck whose leading direction maps in gene space, which is argued to bound prediction reliability.
Key results
- Audit across three encoders and four cell lines under CRISPRi perturbation identifies a rank-1 attention bottleneck in the ST decoder.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| project L0 attention output to rank 1 (ablation) | SE+ST · Replogle-Nadig CRISPRi held-out (mean over 4 cell lines) | mean PDC: 0.374 -> 0.372 | No clear effect |
| project L0 attention output to rank 1 (ablation) | scGPT+ST · Replogle-Nadig CRISPRi held-out (mean over 4 cell lines) | mean PDC: 0.337 -> 0.265 | Hurt |
| replace L0 attention output with a random direction (ablation) | SE+ST · CRISPRi held-out | PDC | Hurt |
| replace output of L1 to L7 with random direction (ablation) | ST decoder · CRISPRi held-out | PDC | No clear effect |
| set top L0 component size to mean over perturbations (ablation) | SE+ST and Tahoe+ST · CRISPRi held-out | PDC | No clear effect |
| swap L0 direction with another perturbation's direction (ablation) | ST decoder · CRISPRi held-out | PDC | Hurt |
| substitute another encoder's direction at L0 (ablation) | ST decoder · CRISPRi held-out | PDC: -0.1 to -0.3 | Hurt |
| project L0 output onto 32 shared training-estimated directions vs random directions (ablation) | ST decoder · CRISPRi held-out | PDC | Improved |
| replace decoder with ridge regression on its inputs | encoder+ST · CRISPRi held-out | mean PDC: within 0.04 PDC | No clear effect |
| evaluate on perturbations far from training | encoder+ST decoders · Replogle-Nadig CRISPRi, far from training (cos dist > 0.5) | mean PDC: 0.07-0.14 vs 0.07 train-mean | Hurt |
| oracle nearest-training-response averaging instead of decoder | oracle retrieval · CRISPRi held-out | PDC: +0.137 PDC on average | Improved |
| use predicted-NN-distance as confidence score | ST decoders · Replogle-Nadig CRISPRi held-out (mean over 4 cell lines) | Spearman |rho| with PDC: 0.88 -> 0.81 | Improved |
Experimental setup
Datasets: CRISPRi perturbation single-cell data (four cell lines)
Builds on: state-transition model (Adduri et al. 2025)
Closest papers by meaning
- PerturBench: Benchmarking Machine Learning Models for Cellular Perturbation Analysis (NeurIPS 2025 · similarity 0.90)
- PertEval-scFM: Benchmarking Single-Cell Foundation Models for Perturbation Effect Prediction (ICML 2025 · similarity 0.88)
- PerturbReason: A Knowledge-Grounded Benchmark and Framework for Cell-State–Conditioned Mechanistic Reasoning of Perturbation Effects (NeurIPS 2026 · similarity 0.87)
- PRESCRIBE: Predicting Single-Cell Responses with Bayesian Estimation (NeurIPS 2025 · similarity 0.87)
- Contextualizing biological perturbation experiments through language (ICLR 2025 · similarity 0.86)
- STRAND: Sequence-Conditioned Transport for Single-Cell Perturbations (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
