Research map / Probabilistic and scientific modeling / Molecules and proteins
GoBERT: Gene Ontology Graph Informed BERT for Universal Gene Function Prediction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GoBERT, a BERT model pre-trained on the Gene Ontology DAG with LLM-encoded term text, combining neighborhood prediction (explicit relations) and a DAG-aware masked-function recovery task (implicit relations) to predict novel gene functions from known annotations.
Problem
Predicting novel functions of genes and gene products using only their known GO annotations, without relying on sequence, structure, or protein family information, while capturing implicit relations between functions.
Approach
Treats GO annotations as tokens in a BERT model, initializes tokens with LLM embeddings of GO term text, pre-trains with a multi-label neighborhood prediction task over the GO DAG adjacency matrix plus an MLM task with a DAG-based masking strategy that excludes roots and nodes having predecessors already annotated.
Key results
- GoBERT reaches 34.08% top-1 and 57.47% top-5 accuracy on novel function prediction, and 53.91% / 76.15% top-1/top-5 accuracy at specified depth, averaged over 5 masking seeds.
- Removing the masking strategy drops top-1 accuracy from 34.08% to 28.03%, the largest ablation drop; removing neighborhood prediction drops it to 30.76%.
- Case study: GoBERT ranks GO:0005739 (mitochondrion) as the most probable cellular component for TOF2 at depth 3 among 5,570 choices.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add neighborhood prediction (ablation) | GoBERT · Novel function prediction | Top-1 accuracy: 30.76 $\pm$ 0.43 -> 34.08 $\pm$ 0.76 | Helped |
| Add neighborhood prediction (ablation) | GoBERT · Novel function prediction | Top-5 accuracy: 53.91 $\pm$ 0.28 -> 57.47 $\pm$ 0.53 | Helped |
| Add neighborhood prediction (ablation) | GoBERT · Novel function prediction | Top-1 accuracy with depth: 50.34 $\pm$ 0.40 -> 53.91 $\pm$ 0.37 | Helped |
| Add neighborhood prediction (ablation) | GoBERT · Novel function prediction | Top-5 accuracy with depth: 73.53 $\pm$ 0.14 -> 76.15 $\pm$ 0.24 | Helped |
| Add LLM-captured semantic information (ablation) | GoBERT · Novel function prediction | Top-1 accuracy: 32.31 $\pm$ 0.48 -> 34.08 $\pm$ 0.76 | Helped |
| Add LLM-captured semantic information (ablation) | GoBERT · Novel function prediction | Top-5 accuracy: 54.96 $\pm$ 0.14 -> 57.47 $\pm$ 0.53 | Helped |
| Add LLM-captured semantic information (ablation) | GoBERT · Novel function prediction | Top-1 accuracy with depth: 51.83 $\pm$ 0.21 -> 53.91 $\pm$ 0.37 | Helped |
| Add LLM-captured semantic information (ablation) | GoBERT · Novel function prediction | Top-5 accuracy with depth: 74.73 $\pm$ 0.13 -> 76.15 $\pm$ 0.24 | Helped |
| Use the designed masking strategy (ablation) | GoBERT · Novel function prediction | Top-1 accuracy: 28.03 $\pm$ 0.49 -> 34.08 $\pm$ 0.76 | Helped |
| Use the designed masking strategy (ablation) | GoBERT · Novel function prediction | Top-5 accuracy: 50.37 $\pm$ 0.30 -> 57.47 $\pm$ 0.53 | Helped |
| Use the designed masking strategy (ablation) | GoBERT · Novel function prediction | Top-1 accuracy with depth: 47.67 $\pm$ 0.33 -> 53.91 $\pm$ 0.37 | Helped |
| Use the designed masking strategy (ablation) | GoBERT · Novel function prediction | Top-5 accuracy with depth: 70.41 $\pm$ 0.06 -> 76.15 $\pm$ 0.24 | Helped |
Limitations
- Only confirmed gene-function annotations are used; negative annotations (gene proven not to have a function) are not modeled.
- Evaluation is limited to a masked-recovery benchmark plus two case studies rather than experimental validation of predicted functions.
- Uses only GO annotations; gene sequence and expression data are not incorporated.
Experimental setup
Datasets: UniEntrezDB Gene Ontology DAG
Benchmarks: novel function prediction benchmark built from UniEntrezDB
Metrics: top-1 accuracy top-5 accuracy top-1 accuracy at depth top-5 accuracy at depth
Baselines: GoBERT w/o Neighborhood Prediction GoBERT w/o Semantic Information GoBERT w/o Masking Strategy InterPro2GO DeepGO DeepFRI
Builds on: BERT GIANT
Closest papers by meaning
- Prot2Text: Multimodal Protein’s Function Generation with GNNs and Transformers (AAAI 2024 · similarity 0.85)
- ProtGO: Function-Guided Protein Modeling for Unified Representation Learning (NeurIPS 2024 · similarity 0.83)
- Prot2Text-V2: Protein Function Prediction with Multimodal Contrastive Alignment (NeurIPS 2025 · similarity 0.83)
- Understanding protein function with a multimodal retrieval-augmented foundation model (NeurIPS 2025 · similarity 0.83)
- Retrieval-Augmented Language Model for Knowledge-aware Protein Encoding (ICML 2025 · similarity 0.83)
- ProtCLIP: Function-Informed Protein Multi-Modal Learning (AAAI 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.