Research map / Large language models / AI and society
WelQrate: Defining the Gold Standard in Small Molecule Drug Discovery Benchmarking
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces WelQrate, a curated collection of 9 PubChem-derived HTS datasets (hierarchical confirmatory/counter screens, PAINS filtering) plus a standardized evaluation framework (metrics, splits, featurization, 3D conformers) and benchmarks of sequence, 2D/3D graph, and descriptor models.
Problem
Existing small molecule virtual screening benchmarks (MoleculeNet, TDC) contain data-quality issues such as inconsistent representations, undefined stereochemistry, experimental artifacts and poorly suited metrics/splits, hindering realistic evaluation of deep learning models for drug discovery.
Approach
Builds a hierarchical, expert-designed curation pipeline over primary, confirmatory and counter HTS screens from PubChem, with PAINS and promiscuity filtering, and pairs it with a standardized evaluation protocol using early-enrichment metrics (logAUC, BEDROC, EF100, DCG100), nested-style random splits and scaffold splits.
Key results
- Domain Baseline (BCL molecular descriptors with MLP) performs best on most metrics except DCG100 in RQ1, outperforming graph and sequence models.
- Models trained on the WelQrate curated data outperform those trained on the uncurated primary-screen control for Naive and sequence-based models across all four metrics.
- Under scaffold splitting (RQ4), all models degrade and advanced 2D/3D graph models lose their advantage over simpler baselines, while the Domain Baseline remains robust.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use a molecular-level BCL descriptor with an MLP instead of the other evaluated representations | WelQrate models · WelQrate, random split | DCG100 | Mixed |
| increase model complexity within the sequence-based, 2D graph-based, or 3D graph-based families | WelQrate models · WelQrate, random split | performance across four metrics | Helped |
| train on the primary-screen-only control data instead of the curated WelQrate data | Naive Baseline and sequence-based models · WelQrate, random split | performance across four metrics | Hurt |
| train on the primary-screen-only control data instead of the curated WelQrate data | 2D and 3D graph-based models · WelQrate, random split | logAUC[0.001,0.1] | Helped |
| train on the primary-screen-only control data instead of the curated WelQrate data | 2D and 3D graph-based models · WelQrate, random split | EF100 | Hurt |
| train on the primary-screen-only control data instead of the curated WelQrate data | Domain Baseline · WelQrate, random split | performance across four metrics | Hurt |
| use one-hot atom-type encoding instead of predefined features | molecular models · AID 1798 | performance | Hurt |
| use one-hot atom-type encoding instead of predefined features | Naive Baseline · AID 1798 | performance | No clear effect |
| use a scaffold split instead of a random split | all evaluated models · WelQrate | performance | Hurt |
| use advanced 2D or 3D graph-based models instead of less complex models | WelQrate models · WelQrate, scaffold split | performance | No clear effect |
| use domain-expert-crafted BCL descriptors | Domain Baseline · WelQrate, scaffold split | performance across all metrics | Helped |
Experimental setup
Datasets: WelQrate datasets (PubChem bioassays AID435008, AID1798, AID435034, AID1843, AID2258, AID463087, AID488997, AID2689, AID MoleculeNet Therapeutics Data Commons
Benchmarks: WelQrate datasets
Metrics: logAUC[0.001,0.1] BEDROC EF100 DCG100
Baselines: SMILES2Vec TextCNN GCN GIN GAT SchNet DimeNet++ SphereNet
Builds on: MoleculeNet Therapeutics Data Commons PAINS filters Lipinski's Rule of Five
Closest papers by meaning
- Data-Driven Discovery of Dynamical Systems in Pharmacology using Large Language Models (NeurIPS 2024 · similarity 0.86)
- Towards precision protein-ligand affinity prediction benchmark: A Complete and Modification-Aware DAVIS Dataset (NeurIPS 2025 · similarity 0.85)
- Monroe: A Molecular Foundation Model for In-context Probabilistic Inference (NeurIPS 2026 · similarity 0.84)
- AANet: Virtual Screening under Structural Uncertainty via Alignment and Aggregation (NeurIPS 2025 · similarity 0.84)
- From Artificially Real to Real: Leveraging Pseudo Data from Large Language Models for Low-Resource Molecule Discovery (AAAI 2024 · similarity 0.84)
- UniMatch: Universal Matching from Atom to Task for Few-Shot Drug Discovery (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.