Research map / Probabilistic and scientific modeling / Molecules and proteins
SpecMER: Fast Protein Generation with K-mer Guided Speculative Decoding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SpecMER, a speculative decoding framework for protein sequence generation that drafts multiple candidates and selects them with k-mer scores from MSAs, yielding higher likelihood and pLDDT while keeping 24-32% speedup over target decoding.
Problem
Autoregressive protein language models are slow at inference, and standard speculative decoding uses draft models that ignore biological/structural constraints, producing less plausible sequences.
Approach
Batch-generate c candidate draft sequences, score each with k-mer frequencies from the wild-type's MSA, and verify only the highest-scoring candidate with standard maximal-coupling speculative decoding, so the target distribution is preserved while guidance biases toward plausible sequences.
Key results
- SpecMER (c=5) reduced top-20 NLL on GFP to 0.41 versus 1.38 for speculative decoding (c=1), with acceptance ratio 0.945 vs 0.911.
- Speculative decoding (c=1) gave 32% speedup over ProGen2-M target-only decoding; SpecMER c=3 gave 29% and c=5 gave 24%.
- Average pLDDT was higher for SpecMER on RBP1 (0.740 at c=5 vs 0.571 for speculative decoding), though GFP was an exception.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add protein-specific MSA-derived k-mer guidance and use 5 candidates | Speculative decoding · GFP, RBP1, ParD3, GB1, Bgl3, ADRB2, CBS | NLL: GFP: 2.45 ± 0.42 -> 1.09 ± 0.64; RBP1: 2.73 ± 0.19 -> 2.41 ± 0.40; ParD3: 1.93 ± 0.59 -> 1.33 ± 0.41; GB1: 2.79 ± 0.15 - | Helped |
| Add protein-specific MSA-derived k-mer guidance and use 5 candidates | Speculative decoding · GFP, RBP1, ParD3, GB1, Bgl3, ADRB2, CBS | acceptance ratio: GFP: 0.911 ± 0.029 -> 0.945 ± 0.084; RBP1: 0.938 ± 0.043 -> 0.926 ± 0.047; ParD3: 0.902 ± 0.036 -> 0.942 ± 0.063; GB1: 0 | Mixed |
| Increase candidate count from c=1 to c=5 | Speculative decoding / SpecMER · GFP, RBP1, GB1 (average) | speedup: 32% -> 24 % | Hurt |
| Increase candidate count from c=1 to c=5 | Speculative decoding / SpecMER · RBP1 | average pLDDT: 0.571 ± 0.116 -> 0.740 ± 0.105 | Helped |
| Use SpecMER rather than speculative decoding | Conditional GFP generation · GFP | average pLDDT: 0.493 ± 0.156 -> 0.426 ± 0.176 | Hurt |
| Use GFP-derived instead of mismatched GB1-derived k-mers (ablation) | SpecMER conditioned on GFP · GFP | Mean NLL: -2.52 ± 0.27; protein-specific result 1.09 ± 0.64 | Helped |
| Use GB1-derived instead of mismatched Bgl3-derived k-mers (ablation) | SpecMER conditioned on GB1 · GB1 | Mean NLL: -2.79 ± 0.10; protein-specific result 2.61 ± 0.27 | Helped |
| Increase Bgl3 MSA depth from 1,000 sequences to the full-depth MSA (ablation) | SpecMER · Bgl3 | Top-20 NLL: 1.56 ± 0.20 -> 0.63 ± 0.11 | Helped |
| Use SpecMER (c=3) instead of target-only decoding | Protein generation with ProGen2-XL target · Protein generation | speedup: 38% | Helped |
| Use SpecMER (c=5) instead of target-only decoding at the same temperatures | Protein generation · GFP, Bgl3, GB1, ParD3 | Top-20 NLL: GFP: 0.51 ± 0.04 -> 0.41 ± 0.07; Bgl3: 0.78 ± 0.02 -> 0.63 ± 0.11; GB1: 2.27 ± 0.24 -> 2.20 ± 0.31; ParD3: 0.69 ± 0.11 - | Helped |
Surprising findings
- SpecMER generated sequences often exceeded the likelihood of target-only decoding, even though it is designed to match the target distribution.
- Cross-protein k-mer ablations (e.g., GFP with GB1 k-mers) reduced likelihoods, indicating gains depend on protein-specific evolutionary k-mers.
Limitations
- Performance depends on MSA quality and depth; sparse or unavailable motifs or disordered proteins may benefit less.
- Batch generation is not truly parallel, and a fully parallel implementation was not attainable on the hardware used.
- Increasing the number of candidates c raises compute and energy cost.
Experimental setup
Datasets: ProteinGym MSAs (GFP, RBP1, ParD3, GB1, Bgl3, ADRB2, CBS)
Benchmarks: ProteinGym MSAs (seven proteins)
Metrics: acceptance ratio NLL top-20 NLL top-5 NLL pLDDT tokens per second speedup WT Hamming distance inter-sequence Hamming distance
Baselines: speculative decoding (c=1) ProGen2-M target-only decoding
Builds on: speculative decoding maximal coupling ProGen2 ProteinGym
Closest papers by meaning
- Sequence-Augmented SE(3)-Flow Matching For Conditional Protein Generation (NeurIPS 2024 · similarity 0.86)
- Steering Protein Language Models (ICML 2025 · similarity 0.85)
- Diffusion on Language Model Encodings for Protein Sequence Generation (ICML 2025 · similarity 0.85)
- Steering Generative Models with Experimental Data for Protein Fitness Optimization (NeurIPS 2025 · similarity 0.85)
- Fast Uncovering of Protein Sequence Diversity from Structure (ICLR 2025 · similarity 0.85)
- LEMON-ZEST: Evolution-Informed Tokenization for Efficient Protein Language Modeling (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.