Research map / Label-efficient and robust learning / Domain adaptation and generalization
Beyond Efficiency: Molecular Data Pruning for Enhanced Generalization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MolPeg, a source-free data pruning framework for molecular transfer learning that scores samples by the absolute loss discrepancy between an online model and an EMA reference model, keeping both easy and hard samples.
Problem
Existing data pruning methods are designed for train-from-scratch in-domain settings and fail when applied to pretrained molecular models, where the source pretraining data is inaccessible and distribution shift between source and target is severe.
Approach
Maintain an online encoder (fast updates on target data) and an EMA reference encoder (slow updates, initialized from the pretrained model) and select samples by the absolute difference of their losses under the two models, retaining both the simplest and hardest samples; the selection is dynamic during training.
Key results
- On HIV, MolPeg reaches 86.0 ROC-AUC at 40% pruning versus 85.1 for full-dataset training.
- On PCBA, MolPeg at 60% pruning reaches 26.4 AP, slightly above the 26.3 full-dataset result, and beats InfoBatch and static baselines at every pruning ratio.
- On QM9 U0 at 60% pruning, MolPeg gives MAE 26.1 meV versus 30.9 for random pruning.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MolPeg instead of InfoBatch | GIN + GraphMAE · HIV | ROC-AUC (%): 84.9 -> 85.5 | Helped |
| Use MolPeg instead of InfoBatch | GIN + GraphMAE · PCBA | Average Precision (%): 25.5 -> 26.4 | Helped |
| Prune 40% of the training data with MolPeg rather than train on the full dataset | GIN + GraphMAE · HIV | ROC-AUC (%): 85.1 -> 86.0 | Helped |
| Prune 60% of the training data with MolPeg rather than train on the full dataset | GIN + GraphMAE · PCBA | Average Precision (%): 26.3$\pm$0.1 -> 26.4 | Helped |
| Use MolPeg rather than random pruning | SchNet · HIV (3D geometry, 40% pruning) | ROC-AUC (%): 80.8 -> 82.3 | Helped |
| Use MolPeg rather than random pruning | PaiNN + GeoSSL · QM9-U0 (90% pruning) | MAE: 85.0 -> 92.4 | Hurt |
| Use MolPeg rather than random pruning | PaiNN + GeoSSL · QM9-Zpve (90% pruning) | MAE: 4.94 -> 5.40 | Hurt |
| Change the pretraining strategy from GraphMAE to GraphCL (ablation) | MolPeg on HIV · HIV | ROC-AUC (%) | Helped |
| Use 3D geometry with SchNet rather than the 2D graph modality (ablation) | MolPeg on HIV · HIV | ROC-AUC (%): 80.8 -> 82.3; 81.4$\pm$1.7 | Mixed |
| Vary the EMA update pace beta across 0.001, 0.01, 0.1, 0.5, and 0.9 (ablation) | MolPeg on HIV · HIV | ROC-AUC (%) | No clear effect |
| Set EMA update pace beta to 0.5 rather than other tested values (ablation) | MolPeg on HIV · HIV | ROC-AUC (%) | Helped |
Surprising findings
- Most existing static data pruning methods underperform even simple random pruning in the source-free pretrain-finetune setting.
- On QM9, all DP methods are worse than random pruning at 80% and 90% pruning ratios, which the authors attribute to close distribution match between PCQM4Mv2 and QM9.
Limitations
- Only validated on molecular tasks; not tested on LLM or vision pretraining settings.
- Simple loss-discrepancy signal; the authors acknowledge better use of source and target knowledge remains open.
- On QM9 with 80-90% pruning, MolPeg and other DP methods underperform random pruning.
Experimental setup
Datasets: PCQM4Mv2 HIV PCBA QM9
Benchmarks: HIV PCBA QM9
Metrics: ROC-AUC Average Precision MAE
Baselines: Hard Random CD Herding K-Means Least Confidence Entropy Forgetting GraNd EL2N DeepFool Craig Glister
Builds on: InfoBatch GraNd GraphMAE SchNet PaiNN
Closest papers by meaning
- Optimal Representation Size: High-Dimensional Analysis of Pretraining and Linear Probing (NeurIPS 2026 · similarity 0.85)
- GDeR: Safeguarding Efficiency, Balancing, and Robustness via Prototypical Graph Pruning (NeurIPS 2024 · similarity 0.84)
- Context-Aware Regularization with Markovian Integration for Attention-Based Nucleotide Analysis (NeurIPS 2025 · similarity 0.84)
- Label-Efficient Dataset Pruning via Semi-Supervised Pseudo-Labeling (NeurIPS 2026 · similarity 0.84)
- DARE the Extreme: Revisiting Delta-Parameter Pruning For Fine-Tuned Models (ICLR 2025 · similarity 0.83)
- Distilling the Knowledge in Data Pruning (ICML 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.