Research map / Probabilistic and scientific modeling / Molecules and proteins
MeToken: Uniform Micro-environment Token Boosts Post-Translational Modification Prediction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Compiles a large sequence-structure PTM dataset and proposes MeToken, which tokenizes residue micro-environments into discrete codebook tokens with uniform sub-codebooks and temperature-scaled VQ to predict PTM types.
Problem
Predicting PTM sites and their specific types from sequence alone ignores tertiary structure, lacks large residue-level sequence-structure PTM datasets, and suffers from a severe long-tail distribution of PTM types.
Approach
Tokenize each residue's sequence-structure micro-environment into discrete tokens via vector quantization, with a uniform sub-codebook loss that allocates equal codebook capacity per PTM type so rare PTMs remain distinguishable, plus temperature-scaled soft-to-hard quantization.
Key results
- On the large-scale dataset, MeToken reaches 50.40 F1 versus 43.94 for fine-tuned ESM-2 (about 6.46 points higher).
- On PTMint, MeToken achieves 42.07 F1 and 98.55 AUROC, outperforming the best baseline MIND (35.04 F1).
- On qPTM, MeToken achieves 61.95 F1 and 88.39 accuracy, above CDConv (45.20 F1) and ESM-2 (57.40 F1).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MeToken instead of fine-tuned ESM-2 | PTM prediction · Large-scale dataset | F1-score: 43.94 -> 50.40 | Helped |
| Use MeToken instead of fine-tuned ESM-2 | PTM prediction · PTMint | F1-score: 36.46 -> 42.07 | Helped |
| Use MeToken instead of fine-tuned ESM-2 | PTM prediction · qPTM | F1-score: 57.40 -> 61.95 | Helped |
| Add micro-environment tokenization to micro-environment modeling (ablation) | MeToken · Large-scale dataset | F1-score: 40.04 -> 50.40 | Helped |
| Add a uniform sub-codebook to micro-environment tokenization (ablation) | MeToken · Large-scale dataset | F1-score: 40.04 -> 48.04 | Helped |
| Replace the uniform sub-codebook with an unconstrained codebook while using temperature-scaled vector quantization (ablation) | MeToken · Large-scale dataset | F1-score: 50.40 -> 37.46 | Hurt |
| Combine AlphaFold2-predicted structures with PDB structures rather than using AlphaFold2-only structures (ablation) | MeToken · Large-scale dataset | F1-score: 11.74 -> 50.40 | Helped |
| Combine AlphaFold2-predicted structures with PDB structures rather than using PDB-only structures (ablation) | MeToken · Large-scale dataset | F1-score: 41.31 -> 50.40 | Helped |
Surprising findings
- Fine-tuned ESM-2 outperforms fine-tuned ESM-3 on PTM prediction in the supplementary experiments.
- Adding temperature-scaled VQ without the uniform sub-codebook lowers F1 (37.46) below the micro-environment model without tokenization (40.04).
Limitations
- Dataset relies predominantly on AlphaFold predictions, which may introduce biases.
- Does not model interactions with enzymes, ligands, or interacting proteins.
- Reported PTMint and qPTM results are from transfer after training on the large-scale dataset, so the cross-dataset comparison depends on that protocol.
Experimental setup
Datasets: dbPTM PDB AlphaFold database PTMint qPTM
Benchmarks: large-scale sequence-structure PTM dataset PTMint qPTM
Metrics: F1-score accuracy precision recall MCC AUROC AUPRC
Baselines: EMBER DeepPhos MusiteDeep ESM-2 StructGNN GraphTrans GVP PiFold GearNet CDConv MIND
Builds on: vector quantization PiGNN GearNet CDConv
Closest papers by meaning
- PROSPECT PTMs: Rich Labeled Tandem Mass Spectrometry Dataset of Modified Peptides for Machine Learning in Proteomics (NeurIPS 2024 · similarity 0.88)
- Improving PTM Site Prediction by Coupling of Multi-Granularity Structure and Multi-Scale Sequence Representation (AAAI 2024 · similarity 0.88)
- Protein Structure Tokenization: Benchmarking and New Recipe (ICML 2025 · similarity 0.86)
- SaProt: Protein Language Modeling with Structure-aware Vocabulary (ICLR 2024 · similarity 0.85)
- ProteinBench: A Holistic Evaluation of Protein Foundation Models (ICLR 2025 · similarity 0.85)
- From Mechanistic Interpretability to Mechanistic Biology: Training, Evaluating, and Interpreting Sparse Autoencoders on Protein Language Models (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.