Research map / Vision-language and multimodal / Human and embodied modeling
Prot2Text-V2: Protein Function Prediction with Multimodal Contrastive Alignment
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Prot2Text-V2 generates free-text protein function descriptions from sequences by projecting ESM-3B embeddings into LLaMA-3.1-8B-Instruct via a two-layer MLP, aligned with a mean/std-pooled contrastive loss (H-SCALE) before LoRA instruction tuning.
Problem
Generating free-form natural language descriptions of protein function directly from amino acid sequences, especially for low-homology proteins where similarity-based methods fail.
Approach
Aligns projected protein sequence embeddings with intermediate (16th-block) decoder text representations using InfoNCE applied separately to mean-pooled and std-pooled features (H-SCALE), before LoRA instruction tuning of a decoder-only LLM, avoiding Q-Former-style modules.
Key results
- Prot2Text-V2 (w/ H-SCALE) reaches exact match 39.16, BLEU-4 43.34, and GPT judge 64.58 on the CD-HIT test split, above Prot2Text (56.95 GPT) and RGCNxESM2LLaMA (58.48 GPT).
- On the ProtT3 split, Prot2Text-V2 reports exact match 49.04 and BLEU-4 57.29 versus ProtT3's 25.74 and 51.47.
- H-SCALE ablation: mean-pooling alignment alone gives BLEU-4 41.23, std-pooling alone 33.78, and combined H-SCALE 43.34 versus 38.98 without alignment.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add H-SCALE contrastive alignment (ablation) | Prot2Text-V2 · SwissProt low-homology test set | B-4: 38.98 -> 43.34 | Helped |
| add H-SCALE contrastive alignment (ablation) | Prot2Text-V2 · SwissProt low-homology test set | GPT: 60.93 -> 64.58 | Helped |
| add mean-pooling alignment (ablation) | Prot2Text-V2 alignment · SwissProt low-homology test set | B-4: 38.98 -> 41.23 | Helped |
| add std-pooling alignment only (ablation) | Prot2Text-V2 alignment · SwissProt low-homology test set | B-4: 38.98 -> 33.78 | Hurt |
| fuse raw sequence and structure encoders with gated cross-attention instead of using the Prot2Text-V2 sequence-to-text approach (ablation) | LLaMA protein function prediction · SwissProt low-homology test set | Exct: 39.16 -> 30.07 | Hurt |
| add the protein name as input (ablation) | Prot2Text-V2 (w/ H-SCALE) · SwissProt test set | B-4: 43.34 -> 52.46 | Helped |
| use Prot2Text-V2 instead of ProtT3 | Protein function generation · ProtT3 original data split | B-4: 51.47 -> 57.29 | Helped |
| use Prot2Text-V2 instead of ProtT3 | Protein function generation · BLAST bitscore [0,30] | B-4: 6.19 -> 11.89 | Helped |
Surprising findings
- Std-pooling-only alignment lowers performance below the no-alignment baseline (BLEU-4 33.78 vs 38.98).
- BLAST outperforms multimodal methods above about 55% bitscore similarity, so neural models only lead in the low-similarity bins.
Limitations
- No wet-lab validation; metrics may not reflect functional correctness.
- Evaluation relies on lexical/semantic metrics and an LLM judge (GPT-4o), with only a small human expert study on nine proteins.
- Comparison with ProtT3 and some baselines is on different splits, complicating direct comparison.
Experimental setup
Datasets: SwissProt (UniProt Release 2025.02, curated multimodal dataset)
Benchmarks: SwissProt CD-HIT split (4,203 test proteins) ProtT3 split
Metrics: exact match BLEU-2 BLEU-4 ROUGE-1 ROUGE-2 ROUGE-L BERTScore RoBERTa BioBERT F1 GPT-4o judge score BLAST bitscore-bin analysis
Baselines: BLAST Prot2Text ProtT3 P2T-GPT GPT-4o-mini Claude-3.5-Sonnet LLaMA-3.1 Galactica SEQ2LLaMA RGCN2LLaMA RGCNxESM2LLaMA
Builds on: ProtT3 Prot2Text InfoNCE LoRA CLIP
Closest papers by meaning
- Prot2Text: Multimodal Protein’s Function Generation with GNNs and Transformers (AAAI 2024 · similarity 0.93)
- ProtCLIP: Function-Informed Protein Multi-Modal Learning (AAAI 2025 · similarity 0.92)
- Understanding protein function with a multimodal retrieval-augmented foundation model (NeurIPS 2025 · similarity 0.91)
- ProtGO: Function-Guided Protein Modeling for Unified Representation Learning (NeurIPS 2024 · similarity 0.90)
- ProSST: Protein Language Modeling with Quantized Structure and Disentangled Attention (NeurIPS 2024 · similarity 0.89)
- MMSite: A Multi-modal Framework for the Identification of Active Sites in Proteins (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.