Research map / Probabilistic and scientific modeling / Molecules and proteins
Prot2Text: Multimodal Protein’s Function Generation with GNNs and Transformers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Prot2Text generates free-text protein function descriptions by fusing an RGCN over AlphaFold structure graphs with ESM2 sequence embeddings and decoding with a pretrained GPT-2, trained on a 256,690-protein SwissProt dataset.
Problem
Protein function prediction is usually framed as multi-label classification over predefined labels, which oversimplifies protein function; the paper reformulates it as free-text generation from sequence and structure.
Approach
Encode the AlphaFold structure as a heterogeneous amino-acid graph with an RGCN and the sequence with ESM2, fuse them via a per-residue graph-conditioned fusion block, and decode with GPT-2 cross-attention.
Key results
- Prot2Text_BASE reaches BLEU 35.11, ROUGE-L 48.49 and BERTScore 84.30 on the test set, beating ESM2-35M alone (BLEU 32.11) and RGCN alone (BLEU 21.63).
- Prot2Text_MEDIUM (398M) gets BLEU 36.51 and Prot2Text_LARGE (898M) gets the best ROUGE-1/2/L and BERTScore (53.68 / 45.60 / 51.40 / 85.20), with inference cost rising with size.
- Concatenating RGCN and ESM2 features without the fusion block performs slightly worse than ESM2 alone (BLEU 30.39 vs 32.11).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the RGCN graph encoder to ESM2-35M and fuse graph and sequence representations per amino acid | Prot2Text_BASE · SwissProt test set | BLEU Score: 32.11 -> 35.11 | Helped |
| Replace the sequence-only ESM2-35M encoder with an RGCN graph encoder | GPT-2 text generation model · SwissProt test set | BLEU Score: 32.11 -> 21.63 | Hurt |
| Use pretrained ESM2-35M instead of a randomly initialized vanilla Transformer | Sequence-only encoder with GPT-2 decoder · SwissProt test set | BLEU Score: 15.75 -> 32.11 | Helped |
| Add RGCN structural information to the randomly initialized vanilla Transformer | vanilla-Transformer encoder with GPT-2 decoder · SwissProt test set | BLEU Score: 15.75 -> 27.97 | Helped |
| Replace per-amino-acid fusion with concatenating the graph representation and sequence representations (ablation) | Prot2Text multimodal encoder · SwissProt test set | BLEU Score: 32.11 -> 30.39 | Hurt |
| Increase the sequence model size from ESM2-8M to ESM2-150M (ablation) | Prot2Text · SwissProt test set | BLEU Score: 30.01 -> 36.51 | Helped |
| Increase the sequence model size from ESM2-150M to ESM2-650M (ablation) | Prot2Text · SwissProt test set | BLEU Score: 36.51 -> 36.29 | Hurt |
| Increase the sequence model size from ESM2-150M to ESM2-650M (ablation) | Prot2Text · SwissProt test set | Rouge-1: 52.13 -> 53.68 | Helped |
| Increase the sequence model size from ESM2-150M to ESM2-650M (ablation) | Prot2Text · SwissProt test set | Inference Time: 1,334 -> 1,667 | Hurt |
Surprising findings
- Simple concatenation of graph and sequence representations (RGCN + ESM2-35M) underperforms ESM2-35M alone, suggesting that adding parameters without fusion can hurt performance.
Limitations
- The RGCN graph encoder is not pretrained, unlike the ESM encoder.
- Test set is filtered to be at most 40% sequence identity to training, so generalization to highly similar proteins is only partly measured.
- Evaluation relies only on n-gram and embedding similarity metrics with no expert or LLM judgment of biological correctness.
Experimental setup
Datasets: SwissProt (UniProtKB release 2022_04) AlphaFold DB structures
Benchmarks: Prot2Text SwissProt test split
Metrics: BLEU ROUGE-1 ROUGE-2 ROUGE-L BERTScore inference time
Baselines: vanilla-Transformer ESM2-35M RGCN RGCN + ESM2-35M RGCN x vanilla-Transformer
Builds on: RGCN ESM2 GPT-2 CD-HIT AlphaFold DB
Closest papers by meaning
- Prot2Text-V2: Protein Function Prediction with Multimodal Contrastive Alignment (NeurIPS 2025 · similarity 0.93)
- ProtGO: Function-Guided Protein Modeling for Unified Representation Learning (NeurIPS 2024 · similarity 0.92)
- ProtCLIP: Function-Informed Protein Multi-Modal Learning (AAAI 2025 · similarity 0.91)
- ProSST: Protein Language Modeling with Quantized Structure and Disentangled Attention (NeurIPS 2024 · similarity 0.89)
- Understanding protein function with a multimodal retrieval-augmented foundation model (NeurIPS 2025 · similarity 0.88)
- MMSite: A Multi-modal Framework for the Identification of Active Sites in Proteins (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.