Research map / Probabilistic and scientific modeling / Molecules and proteins

Whole Genome Transformer for Gene Interaction Effects in Microbiome Habitat Specificity

AAAI 2025 · Probabilistic and scientific modeling › Molecules and proteins · approach: Statistical analysis of ML practice · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Predicts microbial habitat (host, soil, aquatic) from entire genomes by embedding each gene with ESM-2 and training a BERT-like transformer over gene tokens, then extracting gene-pair interactions from high-attention pairs.

Problem

Predicting complex microbial phenotypes such as habitat specificity directly from whole prokaryotic genome sequences, where contig order is unknown and genes are numerous, while also explaining which gene interactions drive the prediction.

Approach

Treat each genome as a set of contigs; embed every gene's protein sequence with ESM-2 (3B), shuffle contigs per epoch, and train an encoder-only transformer with full attention over gene tokens. Attribution clusters top-attention gene pairs and builds gene interaction networks, enabling sequence-level interaction analysis that k-mer and annotation-based methods lack.

Key results

• The model reaches 71% overall accuracy on 1,453 held-out genomes for three-way habitat classification, with host F1 0.82, aquatic F1 0.74, soil F1 0.51.
• Using only about 100 genes from top attributed pairs (k=100) per genome, pseudo-samples still reach 58% accuracy, versus random guessing.
• The model outperforms k-mer baselines: RF 57-59%, linear SVM 56-62%, RBF SVM 63-68%.

Measured effects reported in the paper

Surprising findings

• Soil samples are often misclassified as aquatic (soil recall 0.43), attributed to genomic similarity between soil and aquatic microbiomes.
• Reducing the ESM embedding dimension (ESM2-650M instead of ESM-3B) drops performance to near baseline level, suggesting embedding quality matters more than model depth.

Limitations

• Context length is still memory-limited by full attention; only 4096 gene tokens are used and longer genomes are truncated.
• ESM-2 was trained primarily on eukaryotic proteins, so embeddings may not capture microbial diversity well.
• Analysis is limited to coding regions and to a single phenotype (habitat); many attributed genes lack functional annotation.

Experimental setup

Datasets: ProGenomes v3

Benchmarks: held-out ProGenomes v3 test samples (1,453) pseudo-samples built from attributed genes

Metrics: accuracy precision recall F1

Baselines: random forest on k-mer counts SVM linear on k-mer counts SVM RBF on k-mer counts Traitar

Builds on: ESM-2 BERT FlashAttention DBSCAN

Closest papers by meaning

• Darwin-7B: A Multi-Omic Foundation Model for the Human Gut Microbiome via Sparsified Quality-Aware Tokenization (NeurIPS 2026 · similarity 0.87)
• Modelling Microbial Communities with Graph Neural Networks (ICML 2024 · similarity 0.83)
• Augmenting Biological Fitness Prediction Benchmarks with Landscapes Features from GraphFLA (NeurIPS 2025 · similarity 0.82)
• DNABERT-2: Efficient Foundation Model and Benchmark For Multi-Species Genomes (ICLR 2024 · similarity 0.82)
• MAPE-PPI: Towards Effective and Efficient Protein-Protein Interaction Prediction via Microenvironment-Aware Protein Embedding (ICLR 2024 · similarity 0.81)
• Hyperbolic Genome Embeddings (ICLR 2025 · similarity 0.81)