Research map / Large language models / Language, safety and interpretability
A SARS-CoV-2 Interaction Dataset and VHH Sequence Corpus for Antibody Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces AVIDa-SARS-CoV-2, a labeled VHH–SARS-CoV-2 variant binding dataset from two immunized alpacas, and VHHCorpus-2M (2M+ unique VHH sequences), pre-training VHHBERT and benchmarking it against general protein and antibody language models on binding prediction.
Problem
Pre-trained antibody language models have not been well evaluated for antibody discovery because labeled antigen-binding datasets at the sequence level are scarce; existing SARS-CoV-2 evaluation labels are noisy individual-level disease labels.
Approach
Generate sequence-level binder/non-binder labels for full-length VHHs against 13 SARS-CoV-2 spike antigen types (including Delta, Omicron variants) via phage display plus NGS enrichment statistics, and pair it with a large corpus of full-length VHH sequences for pre-training a VHH-specific RoBERTa model.
Key results
- AntiBERTa2-CSSP achieved the best accuracy (0.854) and F1 (0.652) and AUPRC (0.690) on AVIDa-SARS-CoV-2 binding prediction.
- VHHBERT (pre-trained on VHHCorpus-2M) reached F1 0.608 and AUPRC 0.650, higher than AbLang-H (F1 0.547) despite far fewer pre-training sequences.
- VHHBERT without pre-training had the highest precision (0.811) but much lower recall (0.392) and lowest F1 (0.528), showing pre-training matters.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| pre-train VHHBERT on VHHCorpus-2M rather than use random initialization (ablation) | VHHBERT · AVIDa-SARS-CoV-2 test | F1-score: 0.528~$\pm$~0.008 -> 0.608~$\pm$~0.012 | Helped |
| add CSSP pre-training on human antibody structures | AntiBERTa2 · AVIDa-SARS-CoV-2 test | F1-score: 0.641~$\pm$~0.008 -> 0.652~$\pm$~0.014 | Helped |
| pre-train with antibody sequences instead of general protein sequences | Binding prediction models · AVIDa-SARS-CoV-2 test | AUPRC: 0.584~$\pm$~0.069 -> 0.690~$\pm$~0.011 | Helped |
Surprising findings
- VHHBERT, pre-trained on only 2M sequences, outperformed AbLang-H (14M heavy chains) on F1 and AUPRC and was close to IgBert.
Experimental setup
Datasets: VHHCorpus-2M AVIDa-SARS-CoV-2 AVIDa-hIL6
Benchmarks: AVIDa-SARS-CoV-2
Metrics: accuracy precision recall F1-score AUPRC
Baselines: ProtBert ESM-2 150M ESM-2 650M AbLang-H AntiBERTa2 AntiBERTa2-CSSP IgBert VHHBERT w/o PT
Builds on: RoBERTa ESM-2 AbLang AntiBERTa2 IgBert
Closest papers by meaning
- How to make the most of your masked language model for protein engineering (NeurIPS 2026 · similarity 0.80)
- Immunogenicity Prediction with Dual Attention Enables Vaccine Target Selection (ICLR 2025 · similarity 0.80)
- AsEP: Benchmarking Deep Learning Methods for Antibody-specific Epitope Prediction (NeurIPS 2024 · similarity 0.80)
- ViroGym: Realistic Large-Scale Benchmarks for Evaluating Viral Proteins (NeurIPS 2026 · similarity 0.79)
- A Label Disambiguation-Based Multimodal Massive Multiple Instance Learning Approach for Immune Repertoire Classification (AAAI 2024 · similarity 0.79)
- Synergy of GFlowNet and Protein Language Model Makes a Diverse Antibody Designer (AAAI 2025 · similarity 0.79)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.