Research map / Graph learning / Graph self-supervised learning

Monroe: A Molecular Foundation Model for In-context Probabilistic Inference

NeurIPS 2026 · Graph learning › Graph self-supervised learning · approach: Molecular graph pretraining · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces Monroe, a 58.5M-parameter graph-transformer molecular foundation model pretrained on PM6 quantum properties, conformer denoising and PCBA bioassays, which uses frozen embeddings with TabPFN for in-context prediction and reports state-of-the-art on Polaris and MoleculeACE.

Problem

Bioassay activity prediction is data-limited, and molecular foundation models need to generalize across tasks in low-data regimes, with statistically sound evaluation and sensitivity to activity cliffs.

Approach

Combines a stereochemistry-aware sparse graph transformer (GRIT-based) pretrained on ~81M PM6 molecules plus PCBA with uncertainty-weighted multi-task loss, conformer denoising and embedding decorrelation, and uses a frozen-encoder TabPFN for amortized in-context prediction instead of fine-tuning; the PFN downstream adaptation also improves MiniMol and CheMeleon.

Key results

• Monroe wins or ties for best on 100% of 28 Polaris tasks with mean rank 3.71, versus 5.82 for MiniMol+PFN and 11.46 for CheMeleon+PFN.
• On MoleculeACE, Monroe reaches RMSE 0.629 and cliff RMSE 0.737, the lowest of all methods evaluated.
• Ensembling 10 models (k=10) lowers Polaris mean rank to 2.68 and MoleculeACE cliff RMSE to 0.726.

Surprising findings

• Conformer denoising is not improved by the model itself: predicted geometries are not better than the input conformers under uncertainty weighting.
• Pairwise MoleculeACE differences from stereochemistry augmentation and decorrelation loss were within noise, while Polaris differences were larger.

Limitations

• Monroe is not a good conformation predictor; geometries are not improved over input conformers.
• Cliff RMSE remains well above overall RMSE, so activity cliffs are not solved.
• Benchmark flaws in TDC/Polaris (noisy labels, stereochemical ambiguity) affect the evaluation.

Experimental setup

Datasets: PM6_83M (PubChemQC) PCBA Polaris ADMET subset MoleculeACE

Benchmarks: Polaris (28 tasks) MoleculeACE (30 tasks)

Metrics: win rate mean rank RMSE cliff RMSE AUROC AUPRC

Baselines: MiniMol CheMeleon MolFormer GBM (LightGBM on ECFP4 + RDKit descriptors) MoleculeACE QSAR

Builds on: GRIT TabPFN Uni-Mol+ LoQI VICReg

Closest papers by meaning

• MF-LAL: Drug Compound Generation Using Multi-Fidelity Latent Space Active Learning (ICML 2025 · similarity 0.88)
• Towards Foundational Models for Molecular Learning on Large-Scale Multi-Task Datasets (ICLR 2024 · similarity 0.88)
• Learning Molecular Representation in a Cell (ICLR 2025 · similarity 0.86)
• Knowledge Enhanced Representation Learning for Drug Discovery (AAAI 2024 · similarity 0.85)
• BOOM: Benchmarking Out-Of-distribution Molecular Property Predictions of Machine Learning Models (NeurIPS 2025 · similarity 0.85)
• Towards Fast, Specialized Machine Learning Force Fields: Distilling Foundation Models via Energy Hessians (ICLR 2025 · similarity 0.85)