Research map / Label-efficient and robust learning / Recognition and long-tail
CARE: a Benchmark Suite for the Classification and Retrieval of Enzymes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces CARE, a benchmark suite for enzyme EC-number classification (Task 1) and reaction-to-EC retrieval (Task 2) with OOD splits, and CREEP, a multimodal contrastive baseline (protein, reaction, text) for retrieval.
Problem
No standardized benchmarks exist for evaluating ML models that predict enzyme function (EC numbers) from sequence, or that retrieve enzymes for reactions, especially under out-of-distribution generalization.
Approach
Two tasks with designed OOD train-test splits (sequence identity splits, promiscuous and previously misclassified sets for classification; easy/medium/hard reaction holdouts by EC level for retrieval), plus CREEP, which contrastively aligns protein sequences, reaction SMILES and gene-ontology EC text using finetuned ProtT5, rxnfp and SciBERT encoders.
Key results
- On Task 2 easy split, CREEP (w/ Text) reaches 60.3% level-4 accuracy versus 12.2% for CLIPZyme and 59.3% for the DRFP similarity baseline.
- On Task 1 <30% identity split, CLEAN achieves 55.1% level-4 accuracy, slightly above Foldseek (54.9%) and BLASTp (51.4%).
- On the Task 2 hard split, all methods remain weak, with best level-4 accuracy of only 1.3% (CREEP and CREEP w/ Text).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| added textual EC description (gene ontology text via SciBERT) as third contrastive modality (ablation) | CREEP · Task 2 Easy/Medium/Hard | Level 4 retrieval accuracy: 60.3 vs 39.4 (easy); 7.4 vs 4.1 (medium); 1.3 vs 1.3 (hard) | Helped |
| CREEP (reaction+protein contrastive) versus Similarity Baseline | CREEP · Task 2 Easy | Level 4 retrieval accuracy: 39.4 vs 59.3 | Hurt |
| CREEP (w/ Text) versus Similarity Baseline on harder splits | CREEP (w/ Text) · Task 2 Medium and Hard | Level 3 retrieval accuracy: 59.5 vs 40.2 (medium); 9.8 vs 0.0 (hard) | Helped |
| CLIPZyme graph-based reaction-protein contrastive model used as retrieval baseline | CLIPZyme · Task 2 Easy | Level 4 retrieval accuracy: 12.2 vs 59.3 Similarity Baseline | Hurt |
| Retrieval-based Foldseek and BLASTp vs CLEAN on Task 1 low identity | CLEAN · Task 1 Previously Misclassified (Price) | Level 4 accuracy: 31.8 vs 41.2 (Foldseek) | Hurt |
| Diamond BLASTp nearest-neighbor baseline versus CLEAN on promiscuous split | BLASTp · Task 1 Promiscuous | Level 4 accuracy: 93.7 vs 69.4 (CLEAN) | Helped |
| ChatGPT used for EC prediction from sequence | ChatGPT · Task 1 <30% Identity | Level 4 accuracy: 0.0 vs 55.1 (CLEAN) | Hurt |
Surprising findings
- BLASTp and Foldseek perform close to state-of-the-art CLEAN on Task 1, and BLASTp beats CLEAN on the promiscuous and 30-50% identity splits.
- The Similarity Baseline on Task 2 is strong, outperforming CLIPZyme on the easy split and performing far above random.
Limitations
- Evaluation is at EC-number granularity, which may not capture direct reaction-to-sequence retrieval.
- Some sequence-identity split leakage occurs due to MMseqs2 vs. Diamond BLAST differences, and text-based EC descriptions may introduce indirect leakage.
- Many models (including language models) were unavailable or hard to use, limiting comparisons.
Experimental setup
Datasets: UniProt Swiss-Prot EnzymeMap ECReact Price et al. misclassified set
Benchmarks: CARE Task 1 splits (<30% identity, 30-50% identity, Price, promiscuous) CARE Task 2 splits (easy, medium, hard)
Metrics: top-1 accuracy at EC levels 1-4
Baselines: BLASTp Foldseek CLEAN ChatGPT Pika CLIPZyme Similarity Baseline (DRFP) Random
Builds on: CLIPZyme CLEAN GraphMVP
Closest papers by meaning
- ReactZyme: A Benchmark for Enzyme-Reaction Prediction (NeurIPS 2024 · similarity 0.90)
- CLIPZyme: Reaction-Conditioned Virtual Screening of Enzymes (ICML 2024 · similarity 0.87)
- Retrieval Augmented Zero-Shot Enzyme Generation for Specified Substrate (ICML 2025 · similarity 0.85)
- Generative Enzyme Design Guided by Functionally Important Sites and Small-Molecule Substrates (ICML 2024 · similarity 0.84)
- MMSite: A Multi-modal Framework for the Identification of Active Sites in Proteins (NeurIPS 2024 · similarity 0.84)
- UniZyme: A Unified Protein Cleavage Site Predictor Enhanced with Enzyme Active-Site Knowledge (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.