Research map / Probabilistic and scientific modeling / Molecules and proteins
Training Flexible Models of Genetic Variant Effects from Functional Annotations using Accelerated Linear Algebra
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DeepWAS, which uses banded LD-matrix mini-batches and iterative linear algebra (CoLA: CG and SLQ) to train large Enformer-based neural networks that predict variant effect priors by maximizing the full likelihood of GWAS summary statistics.
Problem
Training large flexible functionally informed priors for genetic variant effects requires inverting and computing log-determinants of huge LD matrices, so prior work has been limited to small models or summary-statistic approximations such as LD score regression.
Approach
Band-approximate the LD matrix so windows act as mini-batches, rewrite the marginal likelihood via Woodbury and the determinant lemma into a well-conditioned matrix B, and compute its inverse and log-determinant with iterative GPU methods (CG, SLQ) so a transformer-CNN prior can be trained by full likelihood.
Key results
- On UK Biobank held-out chromosomes, DeepWAS with the 52M-parameter transformer gave a 3.02% (BMI), 11.70% (height), and 0.694% (asthma) likelihood increase per person, outperforming LDSR, which did not improve with larger models.
- Iterative CG/SLQ on the well-conditioned matrix B was faster than Cholesky on A, even with Nystrom preconditioning, on GPU (A100).
- In semi-synthetic experiments, the transformer DeepWAS model recovered the true f with lower log-space RMSE than constant, GLM, and LDSR-trained models.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use the full LD window rather than a reduced LD window (ablation) | DeepWAS · UK Biobank BMI, Height, and Asthma, held-out chromosomes 6, 7, 8 | percentage likelihood increase per person: 2.38 -> 3.02; 10.32 -> 11.70; 0.358 -> 0.694 | Helped |
| use the full feature window rather than a reduced feature window (ablation) | DeepWAS · UK Biobank BMI, Height, and Asthma, held-out chromosomes 6, 7, 8 | percentage likelihood increase per person: 2.68 -> 3.02; 10.43 -> 11.70; 0.465 -> 0.694 | Helped |
| use the full model size rather than reducing hidden dimension and parameters (ablation) | DeepWAS · UK Biobank BMI, Height, and Asthma, held-out chromosomes 6, 7, 8 | percentage likelihood increase per person: 2.64 -> 3.02; 10.69 -> 11.70; 0.492 -> 0.694 | Helped |
| include the 15 conservation features rather than removing them (ablation) | DeepWAS · UK Biobank BMI, Height, and Asthma, held-out chromosomes 6, 7, 8 | percentage likelihood increase per person: 3.01 -> 3.02; 10.42 -> 11.70; 0.633 -> 0.694 | Helped |
| increase model flexibility from a linear prior to a transformer when trained with LDSR | LDSR · UK Biobank BMI, held-out chromosomes 6, 7, 8 | held-out phenotype prediction quality | Hurt |
| increase model flexibility from a linear prior to a transformer when trained with DeepWAS | DeepWAS · UK Biobank BMI, Height, and Asthma, held-out chromosomes 6, 7, 8 | held-out phenotype prediction quality | Helped |
Surprising findings
- Larger transformer models did not improve, and for BMI slightly decreased, when trained with LD score regression, whereas DeepWAS gained from model size.
- Reducing the LD window, features, or model size often harmed performance, indicating benefit from larger windows and features.
Limitations
- Uses a linear model with a single normal prior, not more flexible priors
- Only a single European population and a single phenotype at a time
- Population stratification handled only via BOLT-LMM covariate-adjusted statistics
Experimental setup
Datasets: UK Biobank BOLT-LMM summary statistics UK Biobank LD matrices ENCODE functional annotations (91 features) FANTOM hCAGE annotations (40 features) PhyloP and phastCons conservation scores dbNSFP coding variant predictions
Benchmarks: semi-synthetic recovery of true f (RMSE in log space) held-out chromosome 6,7,8 UK Biobank likelihood increase for BMI, height, asthma
Metrics: RMSE in log space percentage likelihood increase per person runtime relative gradient error
Baselines: LD score regression constant f model generalized linear model prior Cholesky decomposition Nystrom preconditioned CG
Builds on: LD score regression Enformer CoLA Gaussian process empirical Bayes
Closest papers by meaning
- GV-Rep: A Large-Scale Dataset for Genetic Variant Representation Learning (NeurIPS 2024 · similarity 0.86)
- From Likelihood to Fitness: Improving Variant Effect Prediction in Protein and Genome Language Models (NeurIPS 2025 · similarity 0.86)
- Diet Networks: Thin Parameters for Fat Genomics (ICLR 2017 · similarity 0.85)
- PRSformer: Disease Prediction from Million-Scale Individual Genotypes (NeurIPS 2025 · similarity 0.85)
- Interpreting Genomic Language Models using Sparse Autoencoders (ICML 2026 · similarity 0.84)
- Towards the Efficient Inference by Incorporating Automated Computational Phenotypes under Covariate Shift (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.