Research map / Graph learning / GNN architectures
G-Adapter: Towards Structure-Aware Parameter-Efficient Transfer Learning for Graph Transformer Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Benchmarks traditional PEFTs (Adapter, LoRA, BitFit) on graph transformers and proposes G-Adapter, a structure-aware low-rank adapter using graph-adjacency GraphConv plus Bregman proximal point regularization, matching or beating full fine-tuning with a fraction of trainable parameters.
Problem
Parameter-efficient fine-tuning of pre-trained graph transformer networks is under-explored; traditional language-domain PEFTs lag far behind full fine-tuning on graph tasks due to feature distribution shift.
Approach
Insert a low-rank adapter whose update is a graph convolution, sigma(S X W_down W_up) with S the normalized adjacency (plus self-loops) as structural inductive bias, and regularize fine-tuning with a symmetric-KL Bregman proximal point term that keeps features close to the previous iterate.
Key results
- On MolHIV, G-Adapter reaches 0.790 AUC with 0.24% trainable parameters versus 0.804 AUC for full fine-tuning and 0.763 for LoRA.
- On MolPCBA, G-Adapter reaches 0.269 AP with 1.89% trainable parameters versus 0.272 for full fine-tuning.
- G-Adapter stores a 0.4MB per-task checkpoint versus 161MB for full fine-tuning and roughly halves GPU memory on MolHIV (8932-10658 MB vs 21120 MB).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use G-Adapter instead of LoRA | Graphormer · MolHIV | AUC: 0.763 -> 0.790 | Helped |
| Use G-Adapter instead of LoRA | Graphormer · MolPCBA | AP: 0.246 -> 0.269 | Helped |
| Use G-Adapter instead of full fine-tuning | MAT · FreeSolv | RMSE: 0.286 -> 0.280 | Helped |
| Use G-Adapter instead of full fine-tuning | MAT · ESOL | RMSE: 0.270 -> 0.279 | Hurt |
| Use G-Adapter instead of full fine-tuning | MAT · BBBP | AUC: 0.764 -> 0.750 | Hurt |
| Use G-Adapter instead of full fine-tuning | MAT · Estrogen-β | AUC: 0.778 -> 0.791 | Helped |
| Add graph structure information (ablation) | G-Adapter on Graphormer · MolHIV | AUC: 0.728 -> 0.790 | Helped |
| Add graph structure information (ablation) | G-Adapter on Graphormer · MolPCBA | AP: 0.214 -> 0.269 | Helped |
| Retain Bregman proximal point optimization (ablation) | G-Adapter on Graphormer · MolHIV | AUC: 0.754 -> 0.790 | Helped |
| Retain the pre and post LayerNorm layers (ablation) | G-Adapter on Graphormer · MolHIV | AUC: 0.745 -> 0.790 | Helped |
| Use S2 instead of S1 graph structure information (ablation) | G-Adapter on Graphormer · MolHIV | AUC: 0.790 -> 0.788 | Hurt |
| Use S3 instead of S1 graph structure information (ablation) | G-Adapter on Graphormer · MolPCBA | AP: 0.269 -> 0.250 | Hurt |
Surprising findings
- Graph adjacency-only structure (S1) outperformed distance-based structure (S3), even though distances can be derived from adjacency.
- Original pre-trained parameters yield feature distributions highly similar to full fine-tuning (JS 0.27%), suggesting fine-tuning mainly rescales parameters rather than changing feature encoding.
Limitations
- Evaluated only on molecular graph datasets with two pre-trained GTNs (Graphormer and MAT).
- Introduced modules slow convergence and inference somewhat, with inference overhead noted in the tables.
- Hyperparameters such as the Bregman weight mu and bottleneck size must be tuned per task.
Experimental setup
Datasets: MolHIV MolPCBA FreeSolv ESOL BBBP Estrogen-alpha Estrogen-beta MetStablow MetStabhigh
Benchmarks: MolHIV MolPCBA FreeSolv ESOL BBBP Estrogen-alpha Estrogen-beta MetStablow MetStabhigh
Metrics: AUC AP RMSE trainable parameter ratio GPU memory checkpoint size inference time
Baselines: Full fine-tuning Adapter LoRA BitFit Hyperformer Compacter MAM
Builds on: Adapter LoRA BitFit Graph Convolutional Networks Bregman proximal point optimization
Closest papers by meaning
- AdapterGNN: Parameter-Efficient Fine-Tuning Improves Generalization in GNNs (AAAI 2024 · similarity 0.92)
- Can Classic GNNs Be Strong Baselines for Graph-level Tasks? Simple Architectures Meet Excellence (ICML 2025 · similarity 0.89)
- Classic GNNs are Strong Baselines: Reassessing GNNs for Node Classification (NeurIPS 2024 · similarity 0.89)
- Generalizable Insights for Graph Transformers in Theory and Practice (NeurIPS 2025 · similarity 0.88)
- A Scalable and Effective Alternative to Graph Transformers (AAAI 2025 · similarity 0.88)
- Towards Pre-trained Graph Condensation via Optimal Transport (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.