Research map / 3D vision / 3D generation
CryoBench: Diverse and challenging datasets for the heterogeneity problem in cryo-EM
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces CryoBench, five synthetic cryo-EM datasets with ground truth plus metrics (neighborhood similarity, information imbalance, ARI/AMI, per-image FSC AUC) to benchmark heterogeneous reconstruction methods.
Problem
Heterogeneous cryo-EM reconstruction lacks standardized benchmarks with ground truth and reliable metrics for comparing methods across conformational and compositional heterogeneity.
Approach
Synthetic cryo-EM datasets generated from designed or MD-sampled atomic models (conformational: IgG-1D, IgG-RL, Spike-MD; compositional: Ribosembly, Tomotwin-100) with known ground-truth poses, states and imaging parameters, paired with new latent-embedding and volume metrics for systematic comparison of methods.
Key results
- On IgG-1D, RECOVAR achieved the highest per-image FSC AUC (0.386 mean) among fixed-pose methods.
- On Ribosembly clustering, RECOVAR reached ARI 0.968 and AMI 0.976, while on Tomotwin-100 CryoDRGN reached ARI 0.956.
- Ab initio methods failed to recover the 100 ground-truth structures of Tomotwin-100 (CryoDRGN-AI per-image FSC AUC 0.072).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase noise from SNR 0.01 to SNR 0.005 and 0.001 | heterogeneous reconstruction methods on IgG-1D · IgG-1D with noise | volume metrics | Hurt |
| use the lowest SNR (0.001) | ab initio reconstruction methods on IgG-1D · IgG-1D-noisiest | reconstruction success | Hurt |
Surprising findings
- Fixed-pose CryoDRGN clusters Tomotwin-100 almost perfectly (ARI 0.956) despite fixed poses being an unrealistic setting for complex mixtures.
- Linear methods (3DVA, RECOVAR) struggle to represent small (200 kDa) proteins in Tomotwin-100.
Limitations
- Datasets are synthetic with Gaussian white noise, not capturing realistic noise, junk particles, or non-uniform poses.
- Joint compositional and conformational heterogeneity is not simulated.
- Ab initio methods fail on Tomotwin-100 and at lowest SNR, leaving the benchmark challenging but limited in realism.
Experimental setup
Datasets: IgG-1D IgG-RL Spike-MD Ribosembly Tomotwin-100 EMPIAR-10076 EMPIAR-11247
Benchmarks: IgG-1D IgG-RL Ribosembly Tomotwin-100 Spike-MD
Metrics: Area under per-image FSC curve neighborhood similarity (pMN) information imbalance adjusted rand index adjusted mutual information
Baselines: 3D Classification 3DVA 3DFlex CryoDRGN2 CryoDRGN-AI Opus-DSD RECOVAR
Builds on: cryoDRGN cryoDRGN2 CryoDRGN-AI RECOVAR 3DFlex
Closest papers by meaning
- cryoSPHERE: Single-Particle HEterogeneous REconstruction from cryo EM (ICLR 2025 · similarity 0.93)
- Multiscale guidance of protein structure prediction with heterogeneous cryo-EM data (NeurIPS 2025 · similarity 0.92)
- Reconstructing Heterogeneous Biomolecules via Hierarchical Gaussian Mixtures and Part Discovery (NeurIPS 2025 · similarity 0.92)
- CryoFM: A Flow-based Foundation Model for Cryo-EM Densities (ICLR 2025 · similarity 0.91)
- Mixture of neural fields for heterogeneous reconstruction in cryo-EM (NeurIPS 2024 · similarity 0.91)
- CryoSPIN: Improving Ab-Initio Cryo-EM Reconstruction with Semi-Amortized Pose Inference (NeurIPS 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.