AI Paper Atlas

Research map / Probabilistic and scientific modeling / Molecules and proteins

Towards Understanding the Shape of Representations in Protein Language Models

Kosio Beshkov, Anders Malthe-Sorenssen

ICLR 2026 · Probabilistic and scientific modeling › Molecules and proteins · approach: Biological sequence transformers

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
exclude BOS/EOS token embeddings from shape-space analysis (ablation)ESM2 and Ankh shape space analysis · SCOPe (1377 proteins)effective dimensionMixed
include BOS/EOS token embeddings (ablation)ESM2 and Ankh context sensitivity analysis · SCOPe (1377 proteins)graph filtration momentNo clear effect
increase ESM2 model sizeESM2 · SCOPe, Frechet radiusFrechet radiusNo clear effect
increase ESM2 model sizeESM2 · SCOPe, intermediate layersgraph filtration moment (structural similarity)Improved
increase ESM2 model sizeESM2 · SCOPe, last layergraph filtration moment (structural similarity)No clear effect
go deeper in layersESM2 · SCOPeeffective dimension of shape spaceMixed
use the last layer instead of the layer just before itESM2 · SCOPestructural faithfulnessHurt
vary interpolation spline order and number of sampled points (ablation)SRV shape analysis · ESM2 35Mrobustness of resultsNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy