Research map / Architectures and efficiency / Pruning and efficient tuning
AdapterGNN: Parameter-Efficient Fine-Tuning Improves Generalization in GNNs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AdapterGNN, a parallel-adapter PEFT method for GIN with dual adapters, BatchNorm, and learnable scaling, plus a generalization-bound argument that PEFT tightens error bounds versus full fine-tuning.
Problem
Parameter-efficient fine-tuning methods designed for transformers transfer poorly to GNNs, and full fine-tuning of pre-trained GNNs overfits on small downstream graph datasets.
Approach
Insert two parallel bottleneck adapters into each GIN layer (one on the input before message passing, one on the message-passing output), add tunable batch normalization inside adapters, and use learnable scaling factors initialized small; the frozen pre-trained MLP keeps the expressivity while the small tunable space improves generalization.
Key results
- AdapterGNN averages 71.2% ROC-AUC on eight molecular datasets with 5.2% tuned parameters, versus 70.1% for full fine-tuning.
- On the PPI benchmark AdapterGNN averages 68.9% ROC-AUC with 4.0% tuned parameters, versus 65.2% for full fine-tuning.
- Ablation: removing batch normalization drops average ROC-AUC from 71.2 to 68.0; learnable scaling (71.2) beats the best fixed scaling (69.9).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use AdapterGNN instead of full fine-tuning | GIN fine-tuning · Molecular benchmarks (8 datasets), AttrMasking | ROC-AUC average: 71.1 -> 72.8 | Helped |
| use AdapterGNN instead of full fine-tuning | GIN fine-tuning · PPI | ROC-AUC average: 65.2 -> 68.9 | Helped |
| add a second parallel adapter, so adapters operate before and after message passing (ablation) | AdapterGNN · Molecular benchmarks (6 datasets), AttrMasking | ROC-AUC average: 70.4 -> 71.2 | Helped |
| add batch normalization to the dual parallel adapters (ablation) | AdapterGNN · Molecular benchmarks (6 datasets), AttrMasking | ROC-AUC average: 68.0 -> 71.2 | Helped |
| replace the dual parallel adapters with a sequential adapter after the MLP (ablation) | AdapterGNN · Molecular benchmarks (6 datasets), AttrMasking | ROC-AUC average: 70.2 -> 71.2 | Hurt |
| use learnable scaling instead of fixed scaling (ablation) | AdapterGNN · Molecular benchmarks (6 datasets), AttrMasking | ROC-AUC average: 69.9 -> 71.2 | Helped |
| increase fixed scaling from 0.1 to 5 (ablation) | AdapterGNN · Molecular benchmarks (6 datasets), AttrMasking | ROC-AUC average: 69.9 -> 65.9 | Hurt |
| reduce the adapter bottleneck dimension to limit tunable parameters (ablation) | AdapterGNN · Molecular benchmarks | ROC-AUC | Mixed |
| reduce the training data size (ablation) | AdapterGNN with fewer tunable parameters · Molecular benchmarks | ROC-AUC | Hurt |
Surprising findings
- Larger GIN models generalize worse when trained from scratch, following the classical U-shaped regime rather than the over-parameterized trend seen in large transformers.
- Prompt-based methods (GPF) fall well below full fine-tuning on PPI, with negative transfer frequently observed.
Limitations
- Only evaluated on graph-level classification with GIN backbones; other tasks and GNN architectures are left for future work.
- Theoretical justification relies on a finite-hypothesis-space bound and the assumption that GNNs lie in the classical regime, with several informal steps.
- Inference adds FLOPs and latency relative to full fine-tuning because the adapters are not merged into the backbone.
Experimental setup
Datasets: ZINC15 BACE BBBP ClinTox HIV SIDER Tox21 MUV ToxCast PPI
Benchmarks: BACE BBBP ClinTox HIV SIDER Tox21 MUV ToxCast PPI
Metrics: ROC-AUC generalization gap FLOPs training and inference latency
Baselines: Full fine-tuning Adapter (sequential/parallel) LoRA GPF MolCPT BitFit IA3
Builds on: adapter tuning LoRA BitFit GPF GIN
Closest papers by meaning
- G-Adapter: Towards Structure-Aware Parameter-Efficient Transfer Learning for Graph Transformer Networks (AAAI 2024 · similarity 0.92)
- HG-Adapter: Improving Pre-Trained Heterogeneous Graph Neural Networks with Dual Adapters (ICLR 2025 · similarity 0.91)
- Towards Bridging Generalization and Expressivity of Graph Neural Networks (ICLR 2025 · similarity 0.90)
- Can Classic GNNs Be Strong Baselines for Graph-level Tasks? Simple Architectures Meet Excellence (ICML 2025 · similarity 0.89)
- Classic GNNs are Strong Baselines: Reassessing GNNs for Node Classification (NeurIPS 2024 · similarity 0.89)
- Beyond Weisfeiler-Lehman: A Quantitative Framework for GNN Expressiveness (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.