Research map / Vision-language and multimodal / Cross-modal learning
ProtCLIP: Function-Informed Protein Multi-Modal Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Curates ProtAnno, a 251.5M protein-text pair dataset with property-driven sampling, and pre-trains ProtCLIP with global contrastive, static-segment reconstruction, and property-grouped dynamic segment alignment losses, reporting SOTA on 22 protein benchmarks.
Problem
Existing protein-text multimodal pre-training fails to match the success of image-text foundation models, due to ineffective use of noisy large-scale protein-text pairs and lack of fine-grained, function-informed alignment.
Approach
A property-driven sampling strategy that weights entries by annotation confidence, property coverage, and cluster size to balance quality and quantity, plus a function-informed pre-training paradigm with segment-wise objectives (biotext-guided static segment reconstruction and property-grouped dynamic segment alignment) added to the CLIP loss.
Key results
- ProtCLIP improves GO-CC by 59.9% and GO-BP by 39.7% over baselines on function prediction, per the abstract.
- On cross-modal transformation, ProtCLIP achieves MRR 0.409 on Prot2Drug versus 0.172 for BioBridge, about 2.4x the best baseline.
- Under full tuning, ProtCLIP reaches 95.08% binary and 85.34% subcellular localization accuracy, versus 92.52% and 80.22% for ProtST-ESM2.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ProtAnno-D with property-driven sampling instead of pretrain+finetune (ablation) | ProtCLIP pre-training · DeepLoc Sub | accuracy: 74.98 -> 75.77 | Helped |
| Use ProtAnno-D with property-driven sampling instead of pretrain+finetune (ablation) | ProtCLIP pre-training · EC | AUPR: 0.312 -> 0.384 | Helped |
| Use ProtAnno-D with property-driven sampling instead of pretrain+finetune (ablation) | ProtCLIP pre-training · Prot2MF | MRR: 0.283 -> 0.299 | Helped |
| Add BSR to the full pre-training loss (ablation) | ProtCLIP · EC | AUPR: 0.189 -> 0.204 | Helped |
| Add BSR to the full pre-training loss (ablation) | ProtCLIP · Prot2MF | MRR: 0.282 -> 0.312 | Helped |
| Add PDA to the full pre-training loss (ablation) | ProtCLIP · EC | AUPR: 0.136 -> 0.204 | Helped |
| Add PDA to the full pre-training loss (ablation) | ProtCLIP · Prot2MF | MRR: 0.210 -> 0.312 | Helped |
| Increase PDA threshold to θ ≥ 0.7 (ablation) | ProtCLIP · Sub | performance | Hurt |
| Set loss weights to λ1=0.7 and λ2=0.3 instead of applying no loss weights (ablation) | ProtCLIP pre-training · Pre-training | loss convergence | Helped |
| Use multimodal ProtCLIP rather than ProtST-ESM2 | Protein function classification · GO-BP | AUPR: 0.342 -> 0.567 | Helped |
Surprising findings
- Pretraining on the large noisy ProtAnno-D with the sampling strategy outperforms pretraining on the manually reviewed ProtAnno-S alone, suggesting low-quality data retains value when sampled carefully.
Limitations
- Relies on machine-annotated labels in ProtAnno-D that contain noise and bias.
- Hyperparameters (theta, lambda) selected on a small set of downstream tasks, with some sensitivity reported.
Experimental setup
Datasets: ProtAnno-D ProtAnno-S SwissProt trEMBL PrimeKG
Benchmarks: DeepLoc EC GO PEER beta-lactamase TAPE FLIP PrimeKG cross-modal transformation SHS27K SHS148K STRING
Metrics: accuracy AUPR Fmax Spearman MRR F1
Baselines: ProtST-ESM2 ESM2 ESM-1b ProtBERT OntoProtein BioBridge ComplEx DistMult RotatE KeAP
Builds on: CLIP ProtST BioBridge
Closest papers by meaning
- Prot2Text-V2: Protein Function Prediction with Multimodal Contrastive Alignment (NeurIPS 2025 · similarity 0.92)
- Prot2Text: Multimodal Protein’s Function Generation with GNNs and Transformers (AAAI 2024 · similarity 0.91)
- ProtGO: Function-Guided Protein Modeling for Unified Representation Learning (NeurIPS 2024 · similarity 0.90)
- MMSite: A Multi-modal Framework for the Identification of Active Sites in Proteins (NeurIPS 2024 · similarity 0.89)
- ProSST: Protein Language Modeling with Quantized Structure and Disentangled Attention (NeurIPS 2024 · similarity 0.89)
- Understanding protein function with a multimodal retrieval-augmented foundation model (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.