Research map / Architectures and efficiency / Pruning and efficient tuning
Pin-Tuning: Parameter-Efficient In-Context Tuning for Few-Shot Molecular Property Prediction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Pin-Tuning, a parameter-efficient in-context tuning method for pre-trained molecular encoders in few-shot molecular property prediction, combining bottleneck adapters with context-conditioned inputs and Bayesian (Fisher-based) weight consolidation on embedding layers.
Problem
Fine-tuning pre-trained molecular encoders in few-shot molecular property prediction performs worse than freezing them, due to parameter-data imbalance (overfitting and forgetting) and the encoder's lack of awareness of molecular context.
Approach
Insert a lightweight bottleneck adapter into frozen pre-trained message passing layers, feed it molecular-context vectors (from a GNN context encoder over a molecule-property graph) so the encoder adapts in-context, and regularize pre-trained atom/bond embeddings with a Bayesian/Fisher-weighted L2 penalty (Emb-BWC).
Key results
- On Tox21 10-shot, Pin-Tuning reaches 91.56 ROC-AUC versus 86.67 for GS-Meta, and on SIDER 10-shot it reaches 93.41 versus 84.36 (+10.73%).
- With d2=50 on Tox21, Pin-Tuning trains only about 14.2% of the parameters needed by full fine-tuning (GS-Meta).
- Ablation: removing the bottleneck adapter drops Tox21 10-shot from 91.56 to 79.72 ROC-AUC.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the MP-Adapter bottleneck module (ablation) | Pin-Tuning · Tox21 10-shot | ROC-AUC (%): 79.72 -> 91.56 | Helped |
| Add molecular context to the adapter (ablation) | Pin-Tuning · SIDER 10-shot | ROC-AUC (%): 74.68 -> 93.41 | Helped |
| Add LayerNorm to the adapter (ablation) | Pin-Tuning · Tox21 10-shot | ROC-AUC (%): 86.71 -> 91.56 | Helped |
| Increase adapter hidden dimension to 50 (ablation) | Pin-Tuning · Tox21 | ROC-AUC (%) | Helped |
| Vary the Emb-BWC regularizer weight across {0.01, 0.1, 1, 10} (ablation) | Pin-Tuning · Tox21 | ROC-AUC (%) | Mixed |
| Use the identity-matrix Emb-BWC approximation instead of the other regularizer approximations (ablation) | Pin-Tuning · Few-shot molecular property prediction | ROC-AUC (%) | Helped |
| Use Emb-BWC regularization when fine-tuning embedding layers (ablation) | Pin-Tuning · Few-shot molecular property prediction | ROC-AUC (%) | Helped |
Limitations
- Reported standard deviations are higher than baselines on several datasets (e.g., 2.57 on Tox21 10-shot)
- Results rely on a single pre-trained GIN encoder (Pre-GNN) and five MoleculeNet datasets
Experimental setup
Datasets: Tox21 SIDER MUV ToxCast PCBA
Benchmarks: Tox21 SIDER MUV ToxCast PCBA
Metrics: ROC-AUC
Baselines: Siamese ProtoNet MAML TPN EGNN IterRefLSTM Pre-GNN Meta-MGNN PAR GS-Meta
Builds on: MAML Pre-GNN GS-Meta elastic weight consolidation adapter tuning
Closest papers by meaning
- MIPT: Multilevel Informed Prompt Tuning for Robust Molecular Property Prediction (ICML 2025 · similarity 0.89)
- From Molecules to Materials: Pre-training Large Generalizable Models for Atomic Property Prediction (ICLR 2024 · similarity 0.86)
- TRIDENT: Tri-Modal Molecular Representation Learning with Taxonomic Annotations and Local Correspondence (NeurIPS 2025 · similarity 0.86)
- Physical Consistency Bridges Heterogeneous Data in Molecular Multi-Task Learning (NeurIPS 2024 · similarity 0.85)
- Association Pattern-enhanced Molecular Representation Learning (AAAI 2025 · similarity 0.85)
- Monroe: A Molecular Foundation Model for In-context Probabilistic Inference (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.