Research map / Large language models / Agents, code and math

PerturbReason: A Knowledge-Grounded Benchmark and Framework for Cell-State–Conditioned Mechanistic Reasoning of Perturbation Effects

NeurIPS 2026 · Large language models › Agents, code and math · approach: LLMs for scientific discovery

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces PertReasonQA, a knowledge-grounded QA benchmark for cell-state-conditioned mechanistic reasoning about single-cell perturbation effects, and PertReasonLM (Qwen3-4B, SFT then GRPO) as a reference probe.

Problem

Perturbation-response models are judged only on outcome accuracy, so they can be right for wrong reasons; there is no benchmark that checks whether predicted expression changes are supported by context-specific mechanistic pathways under distribution shift.

Approach

Pairs single-cell CRISPRi and chemical perturbation outcomes with knowledge-graph reasoning paths conditioned on each cell's basal expression (sigmoid gating of edge weights, KS-derived low/high thresholds), filters paths by sign-consistency with the observed label, and trains an LLM with SFT plus GRPO to output outcome, reasoning triplets and text.

Key results

• PertReasonLM-4B GRPO reaches 0.736 average balanced accuracy across eight test folds, versus 0.367 for Qwen3-4B 0-shot and 0.411 for STATE.
• Under the default cell-conditioned path context, PertReasonLM-GRPO achieves 0.976 edge recall and 0.938 GO similarity.
• CoT SFT raises average balanced accuracy from 0.636 (outcome-only SFT) to 0.709; GRPO reduces the answer-correct/reason-wrong failure mode from 2.1% to 0.8%.

Surprising findings

• Proprietary GPT-5.4-full reaches balanced accuracy (0.380-0.399) comparable to Qwen3-4B (0.368) on the mini benchmark, suggesting scale alone does not give context-specific reasoning.
• Adding unconditional (global KG) paths degrades prediction performance relative to no-path context.

Limitations

• Reference paths are curated KG-derived proxies, not exhaustive biological ground truth; KGs are incomplete and biased toward well-studied genes.
• Reasoning texts are synthesized by Qwen3-4B, which may introduce stylistic bias favoring similar models.
• Models struggle to arbitrate among competing mechanisms implying different outcome directions in the conflict subset.

Experimental setup

Datasets: CRISPRi genetic perturbation datasets chemical perturbation datasets (four large-scale sources) OmniPath MSigDB INDRA CoGEx

Benchmarks: PertReasonQA PertReasonQA-mini

Metrics: balanced accuracy edge recall path connectivity gene ontology similarity answer-reason taxonomy

Baselines: GEARS scGPT STATE BioMistral 7B NatureLM 8x7B SUMMER Qwen3-4B GPT-5.4

Builds on: TieDIE GRPO PerturbQA SUMMER

Closest papers by meaning

• Contextualizing biological perturbation experiments through language (ICLR 2025 · similarity 0.91)
• PerturBench: Benchmarking Machine Learning Models for Cellular Perturbation Analysis (NeurIPS 2025 · similarity 0.90)
• Cradle-VAE: Enhancing Single-Cell Gene Perturbation Modeling with Counterfactual Reasoning-based Artifact Disentanglement (AAAI 2025 · similarity 0.87)
• STRAND: Sequence-Conditioned Transport for Single-Cell Perturbations (NeurIPS 2026 · similarity 0.87)
• A benchmark for prediction of transcriptomic responses to chemical perturbations across cell types (NeurIPS 2024 · similarity 0.86)
• MapPFN: Learning Causal Perturbation Maps in Context (NeurIPS 2026 · similarity 0.86)