Research map / Graph learning / GNN architectures
Accelerating Molecular Graph Neural Networks via Knowledge Distillation
Filip Ekström Kelvinius, Dimitar Georgiev, Artur Toshev, Johannes Gasteiger
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add n2n feature distillation from PaiNN | PaiNN-small · OC20 S2EF validation | Energy MAE: 489 -> 457 | Improved |
| Add n2n feature distillation from PaiNN | PaiNN-small · OC20 S2EF validation | Force MAE: 47.1 -> 46.7 | Improved |
| Add n2n feature distillation from GemNet-OC | GemNet-OC-small · OC20 S2EF validation | Energy MAE: 344 -> 310 | Improved |
| Add e2e feature distillation from GemNet-OC | GemNet-OC-small · OC20 S2EF validation | Force MAE: 31.3 -> 29.7 | Improved |
| Add n2n feature distillation from GemNet-OC | PaiNN · OC20 S2EF validation | Energy MAE: 440 -> 346 | Improved |
| Add e2n feature distillation from GemNet-OC | PaiNN · OC20 S2EF validation | Force MAE: 45.3 -> 41.3 | Improved |
| Add vector-to-vector distillation from PaiNN | PaiNN-small · COLL test set | Force MAE: 80.9 -> 70.4 | Improved |
| Add n2n feature distillation from GemNet-OC | PaiNN · COLL test set | Energy MAE: 85.8 -> 60.4 | Improved |
| Use feature-based n2n distillation instead of prediction-level Vanilla (1) | PaiNN-small · COLL test set | Energy MAE: Vanilla (1): 106.1; n2n: 92.5 | Mixed |
| Replace MSE feature loss with LSP, GSP, or direct CKA optimization (ablation) | GemNet-OC teacher to PaiNN student on OC20 · OC20 S2EF validation | predictive accuracy | Hurt |
| Replace the learned linear student projection with other transformation functions (ablation) | GemNet-OC teacher to PaiNN student · OC20 S2EF validation | predictive accuracy | Hurt |
| Distill features from earlier teacher layers or into earlier student layers (ablation) | PaiNN student with GemNet-OC teacher · OC20 S2EF validation | predictive accuracy | Hurt |
Closest papers by meaning
- On the Scalability of GNNs for Molecular Graphs (NeurIPS 2024 · similarity 0.89)
- Pre-Training Graph Neural Networks on Molecules by Using Subgraph-Conditioned Graph Information Bottleneck (AAAI 2025 · similarity 0.87)
- MAGE: Model-Level Graph Neural Networks Explanations via Motif-based Graph Generation (ICLR 2025 · similarity 0.87)
- Pushing the Limits of All-Atom Geometric Graph Neural Networks: Pre-Training, Scaling, and Zero-Shot Transfer (ICLR 2025 · similarity 0.87)
- Neural Atoms: Propagating Long-range Interaction in Molecular Graphs through Efficient Communication Channel (ICLR 2024 · similarity 0.87)
- MIPT: Multilevel Informed Prompt Tuning for Robust Molecular Property Prediction (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
