Research map / Probabilistic and scientific modeling / Variational and generative sampling
Protein Discovery with Discrete Walk-Jump Sampling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Discrete Walk-Jump Sampling, which trains an energy-based model on noise-smoothed one-hot antibody sequences, samples with Langevin MCMC, and denoises in one step, and introduces the distributional conformity score for evaluating protein generative models.
Problem
Training and sampling from discrete generative models is brittle for EBMs and diffusion models, and protein (antibody) discovery needs sample-efficient, non-autoregressive generators evaluated by more than sequence recovery.
Approach
Smooth discrete data with a single Gaussian noise level, train an EBM by contrastive divergence on noisy data, sample with Langevin MCMC ("walk") and project back to the data manifold with a separately trained one-step least-squares denoiser ("jump"); the decoupling lets every chain sample be valid.
Key results
- dWJS energy-based achieved 0.70 total binding rate in the first wet-lab round on HER2 versus 0.25 for LaMBO-2 and 0.22 for AbDiffuser.
- dWJS score-based achieved 1.0 total expressed fraction in the wet lab, and 270 of 277 designed antibodies were expressed and purified (97.47%).
- dWJS energy-based reached DCS 0.38 and score-based 0.49 on the in silico benchmark, with energy-based sampling about 43x faster than IgLM.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use dWJS (energy-based) instead of clean-data EBM sampling | antibody generation · in vitro antibody expression | total_expressed: 0.42 -> 0.97 | Helped |
| use dWJS (energy-based) instead of diffusion baselines | HER2 antibody design · HER2 binding, unique designs | total_bind: 0.25 -> 0.70; 0.22 (0.57) -> 0.70 | Helped |
| use dWJS (score-based) instead of SeqVDM | paired OAS antibody generation · paired OAS | DCS: 0.40 -> 0.49 | Helped |
| use dWJS (energy-based) instead of IgLM | paired OAS antibody generation · paired OAS | W_property: 0.08 -> 0.056 | Helped |
| use dWJS (energy-based) instead of IgLM | paired OAS antibody generation · paired OAS | DCS: 0.533 -> 0.38 | Hurt |
| set noise level σ to 0.1 instead of 0.5 (ablation) | score-based dWJS · paired OAS antibody generation | W_property: 0.065 -> 0.378 | Hurt |
| set noise level σ to 0.1 instead of 0.5 (ablation) | score-based dWJS · paired OAS antibody generation | E_dist: 62.7 -> 120.6* | Mixed |
| set noise level σ to 3.0 instead of 0.5 (ablation) | score-based dWJS · paired OAS antibody generation | IntDiv: 65.1 -> 30.0 | Hurt |
| set noise level σ to 3.0 instead of 0.5 (ablation) | score-based dWJS · paired OAS antibody generation | W_property: 0.065 -> 0.130 | Hurt |
Surprising findings
- Sampling with the EBM walk and a denoiser jump gave better samples than using the denoiser for both walk and jump (score-based dWJS) on some metrics, despite the EBM being a separate model.
- ESM2 scored a high edit distance and internal diversity but very low DCS (0.061), showing those diversity metrics can be misleading for non-generative models.
Limitations
- Evaluated mainly on antibody sequences; generality to other molecules or modalities is future work.
- DCS relies on kernel density estimation over a small set of biophysical and property features and is not validated on all protein classes.
- Wet-lab binding evaluation covers a single target (HER2) and a single trastuzumab CDR H3 redesign task.
Experimental setup
Datasets: Observed Antibody Space (OAS) hu4D5 antibody mutant dataset (binder set)
Benchmarks: OAS antibody-likeness metrics hu4D5 HER2 binding classifier
Metrics: distributional conformity score W_property unique fraction edit distance internal diversity probability of binding expression rate binding rate
Baselines: SeqVDM DEEN IgLM ESM2 GPT 3.5 GPT 4 LaMBO-2 AbDiffuser BERT transformer EGNN EBM
Builds on: neural empirical Bayes walk-jump sampling contrastive divergence variational diffusion models DEEN
Closest papers by meaning
- Action-Minimization Meets Generative Modeling: Efficient Transition Path Sampling with the Onsager-Machlup Functional (ICML 2025 · similarity 0.88)
- SE(3)-Stochastic Flow Matching for Protein Backbone Generation (ICLR 2024 · similarity 0.88)
- Steering Masked Discrete Diffusion Models via Discrete Denoising Posterior Prediction (ICLR 2025 · similarity 0.88)
- Antibody Design Using a Score-based Diffusion Model Guided by Evolutionary, Physical and Geometric Constraints (ICML 2024 · similarity 0.88)
- AlphaFold Meets Flow Matching for Generating Protein Ensembles (ICML 2024 · similarity 0.88)
- Learning conformational ensembles of proteins based on backbone geometry (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.