Research map / Probabilistic and scientific modeling / Molecules and proteins
Removing Biases from Molecular Representations via Information Maximization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes InfoCORE, a reweighted InfoNCE objective that lower-bounds conditional mutual information given a batch identifier, to remove batch effects from molecular representations learned from drug screens and to enforce fairness.
Problem
Batch effects in high-throughput drug screening confound multimodal molecule-phenotype representations, and existing conditional contrastive methods struggle with limited negatives per batch.
Approach
Derives a variational lower bound on conditional mutual information I(Z_d;Z_g|X_b) that yields an InfoNCE loss whose negatives are reweighted by estimated posterior batch distributions, combined with a batch classifier loss, so that the full negative pool is used while batch information is suppressed.
Key results
- On gene expression retrieval over the batch library, InfoCORE reaches 14.03% top-1 vs 12.23% for CLIP and 12.76% for CCL.
- Average ROC-AUC over seven MoleculeNet classification tasks is 78.3 for InfoCORE with GE pretraining vs 77.6 for CLIP and CCL.
- On UCI Adult, InfoCORE reduces equalized odds to 14.9 vs 20.7 for CLIP while keeping accuracy at 85.2.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use InfoCORE instead of CLIP | Molecular retrieval · GE whole | Top 1 accuracy (%): 5.96 -> 6.39 | Helped |
| Use InfoCORE instead of CCL | Molecular retrieval · GE batch | Top 1 accuracy (%): 12.76 -> 14.03 | Helped |
| Use InfoCORE instead of CLIP | Molecular retrieval · CP whole | Top 1 accuracy (%): 7.23 -> 6.93 | Hurt |
| Use InfoCORE instead of CCL | Molecular retrieval · CP batch | Top 5 accuracy (%): 37.99 -> 38.50 | Helped |
| Pretrain with InfoCORE rather than CLIP | Molecular property prediction · MoleculeNet classification average (GE pretraining) | ROC-AUC (%): 77.6 -> 78.3 | Helped |
| Pretrain with InfoCORE rather than CCL | Molecular property prediction · MoleculeNet classification average (CP pretraining) | ROC-AUC (%): 77.5 -> 78.0 | Helped |
| Use InfoCORE rather than CCL | Fair molecular representations · UCI Adult | EO: 19.0(3.3) -> 14.9(1.1) | Helped |
| Use InfoCORE rather than CCL | Fair molecular representations · Law School | EO: 27.8(1.5) -> 25.4(3.6) | Helped |
| Use InfoCORE rather than CCL | Fair molecular representations · Compas | EOPP: 10.1(3.0) -> 9.3(0.8) | Helped |
| Set α=1, removing posterior-based negative-sample reweighting (ablation) | InfoCORE · Contrastive learning objective | conditional mutual information lower bound | Mixed |
Surprising findings
- CCL performs poorly on whole-library retrieval (e.g., 1.93% top-1 on GE) because it relies on few negatives within the same batch.
Limitations
- Gains over CLIP and CCL in several property prediction tasks are small and sometimes within standard deviation.
- Requires a categorical batch identifier and an alpha tradeoff hyperparameter plus a gradient schedule lambda.
Experimental setup
Datasets: LINCS L1000 gene expression Cell Painting (CellProfiler features) MoleculeNet PRISM UCI Adult Law School Compas
Benchmarks: molecule-phenotype retrieval (GE and CP) MoleculeNet classification (BBBP, BACE, ClinTox, Tox21, ToxCast, SIDER, HIV) PRISM regression UCI Adult Law School Compas
Metrics: top-N retrieval accuracy ROC-AUC R^2 accuracy equalized odds equality of opportunity demographic parity KNN purity batch mixing entropy
Baselines: CLIP CCL Mol2vec SimCLR-based CLIP and InfoCORE variants
Builds on: InfoNCE CLIP conditional contrastive learning (CCL) MoCo Mol2vec
Closest papers by meaning
- Learning Molecular Representation in a Cell (ICLR 2025 · similarity 0.87)
- Monroe: A Molecular Foundation Model for In-context Probabilistic Inference (NeurIPS 2026 · similarity 0.85)
- A benchmark for prediction of transcriptomic responses to chemical perturbations across cell types (NeurIPS 2024 · similarity 0.84)
- Knowledge Enhanced Representation Learning for Drug Discovery (AAAI 2024 · similarity 0.84)
- UniMatch: Universal Matching from Atom to Task for Few-Shot Drug Discovery (ICLR 2025 · similarity 0.84)
- Learning Identifiable Factorized Causal Representations of Cellular Responses (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.