Research map / Label-efficient and robust learning / Self-supervised and contrastive

Distilling Structural Representations into Protein Sequence Models

ICLR 2025 · Label-efficient and robust learning › Self-supervised and contrastive · approach: Self-supervised speech and brain decoding · 11 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Distills structural representations from protein structure models into protein sequence models (ESM2) to yield representations capturing both evolutionary and structural properties.

Problem

Obtaining protein representations that capture both sequence evolutionary history and atomic structural properties, which sequence models like ESM2 lack.

Approach

Implicit Structure Model (ISM) distills structural information into a protein sequence model so that it captures structural properties without requiring structure input at inference; based on the abstract, the approach is a distillation of structural representations into ESM2-style sequence models.

Experimental setup

Builds on: ESM2

Closest papers by meaning

• LEMON-ZEST: Evolution-Informed Tokenization for Efficient Protein Language Modeling (NeurIPS 2026 · similarity 0.89)
• From Mechanistic Interpretability to Mechanistic Biology: Training, Evaluating, and Interpreting Sparse Autoencoders on Protein Language Models (ICML 2025 · similarity 0.89)
• DPLM-2: A Multimodal Diffusion Protein Language Model (ICLR 2025 · similarity 0.89)
• Understanding protein function with a multimodal retrieval-augmented foundation model (NeurIPS 2025 · similarity 0.88)
• Metalic: Meta-Learning In-Context with Protein Language Models (ICLR 2025 · similarity 0.88)
• Diffusion Language Models Are Versatile Protein Learners (ICML 2024 · similarity 0.88)