Research map / Probabilistic and scientific modeling / Molecules and proteins
Adaptive Protein Tokenization
Rohit Dilip, Ayush Varshney, David Van Valen
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| drop the token tail (keep 16 instead of 128 tokens) (ablation) | APT autoregressive generation · Unconditional generation | designability: 0.232 -> 0.482 | Improved |
| add classifier annealing (w=0.8) to 16-token generation (ablation) | APT · Unconditional generation | designability: 0.482 -> 0.771 | Improved |
| entropy-based tail cutoff instead of fixed cutoff, without classifier annealing (ablation) | APT · Unconditional generation | designability: 0.482 -> 0.494 | No clear effect |
| entropy-based cutoff combined with classifier annealing (ablation) | APT · Unconditional generation | designability: 0.771 -> 0.871 | Improved |
| strengthen classifier annealing to w=0.3 (ablation) | APT · Unconditional generation | designability / TM to unannealed: designability 0.462 -> 0.669; TM 0.93 -> 0.15 | Mixed |
| keep more tokens in generation (ablation) | APT · Unconditional generation | designability vs gFID | Mixed |
| scale FSQ codebook from 1000 to 4000+ (ablation) | APT tokenizer · Reconstruction | RMSD | Improved |
| use wider (384) and deeper (4 layer) encoder (ablation) | APT 4k codebook · Reconstruction | RMSD and flow loss | Hurt |
| train AR model on tokenizer with 128 vs 64 max tokens (ablation) | APT AR model · Unconditional generation | designability | No clear effect |
| use MLP probe instead of linear probe on APT tokens (ablation) | APT representations · CATH-T classification | top-1 accuracy | Improved |
| lower flow loss during training (ablation) | APT training · Reconstruction | RMSD | Hurt |
Closest papers by meaning
- Protein Structure Tokenization: Benchmarking and New Recipe (ICML 2025 · similarity 0.92)
- LEMON-ZEST: Evolution-Informed Tokenization for Efficient Protein Language Modeling (NeurIPS 2026 · similarity 0.90)
- Elucidating the Design Space of Multimodal Protein Language Models (ICML 2025 · similarity 0.86)
- Diffusion on Language Model Encodings for Protein Sequence Generation (ICML 2025 · similarity 0.86)
- FoldToken: Learning Protein Language via Vector Quantization and Beyond (AAAI 2025 · similarity 0.86)
- Atom-level Protein Representation Learning Improves Protein Structure Prediction (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
