Research map / Graph learning / GNN architectures
Fine-Tuning Graph Neural Networks by Preserving Graph Generative Patterns
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes G-Tuning, a GNN fine-tuning strategy that reconstructs the graphon (generative pattern) of downstream graphs using a linear combination of learnable graphon bases, with a Gromov-Wasserstein loss, to mitigate structural divergence between pre-training and downstream data.
Problem
Pre-trained GNNs transfer poorly when the structure of pre-training and downstream graphs diverges (negative transfer), especially out-of-domain, because the generative patterns differ.
Approach
Fine-tune the pre-trained GNN so it reconstructs the graphon of each downstream graph, approximated as a linear combination of learnable graphon bases whose coefficients are predicted by a GNN decoder from embeddings and adjacency, supervised by a GW-discrepancy loss against an oracle graphon estimated via structured Gromov-Wasserstein barycenters.
Key results
- Average rank 1.5 on 8 in-domain molecular datasets (ROC-AUC), best on 6 of 8 datasets, versus 5.8 for vanilla tuning and 3.2 for GTOT-Tuning.
- Out-of-domain (GCC pre-training) on 7 TUDataset benchmarks: best on 6 of 7, average rank 1.1; improves PROTEINS by 7.63% over vanilla tuning.
- Reported average improvement of 0.47% (in-domain) and 2.62% (out-of-domain) over the best baseline.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use G-Tuning instead of the strongest reported baseline | GIN with context-prediction pre-training · In-domain molecular benchmarks (8 datasets) | Average rank: 1.5 vs. 3.2 | Helped |
| Use G-Tuning instead of the strongest reported baseline | GCC-pre-trained model · Cross-domain graph benchmarks (7 datasets) | Accuracy: 6 out of 7 datasets; MUTAG 1.92% lower | Mixed |
| Add graphon reconstruction loss to GIN without pre-training (ablation) | GIN · PROTEINS and ENZYMES (original node features) | Accuracy: 73.3±4.0 -> 74.9±3.1; 59.6±4.5 -> 66.0±6.2 | Helped |
| Add pre-training to G-Tuning (ablation) | G-Tuning · In-domain benchmarks | ROC-AUC | Mixed |
| Use graphon decomposition rather than direct graphon reconstruction (ablation) | G-Tuning · Four datasets in the ablation study | Performance | Helped |
| Use a structural GNN encoder rather than an MLP encoder (ablation) | G-Tuning graphon reconstruction module · Ablation study datasets | Performance | Helped |
| Use Gromov-Wasserstein discrepancy rather than KL divergence, Wasserstein distance, or cosine similarity (ablation) | G-Tuning graphon reconstruction loss · Ablation study datasets | Performance | Helped |
| Increase the number of graphon bases from 2 to 32 (ablation) | G-Tuning · Hyperparameter study | Performance | Helped |
| Increase the number of graphon bases beyond 32 (ablation) | G-Tuning · Hyperparameter study | Performance | No clear effect |
| Constrain embeddings with Feature-Map or DELTA rather than use vanilla tuning | Fine-tuning GNNs · Some in-domain molecular datasets | ROC-AUC | Hurt |
| Use GTOT-Tuning rather than vanilla tuning | GCC-pre-trained model · PROTEINS and ENZYMES | Accuracy | Hurt |
Surprising findings
- Graphon discrepancy between pre-training and downstream graphs is the only graph measurement consistently negatively correlated with performance gain among those tested.
- Several computer-vision fine-tuning regularizers (Feature Map, DELTA) sometimes perform worse than vanilla tuning on graph data.
Limitations
- Oracle graphon estimation dominates the extra running time, and running time grows with the number of bases.
- Performance gains are small in-domain (about 0.5% on average), and the method is not clearly better on some datasets (e.g., MUTAG, ENZYMES cases).
- Out-of-domain experiments use structural node features from GCC rather than original features, so results are not directly comparable to prior reports.
Experimental setup
Datasets: ZINC15 MoleculeNet (BBBP, Tox21, Toxcast, SIDER, ClinTox, MUV, HIV, BACE) TUDataset (IMDB-M, IMDB-B, MUTAG, PROTEINS, ENZYMES, MSRC_21, RDT-M12K) GCC pre-training graphs (Academia, DBLP, IMDB, Michigan, UIllinois, MSU, Wiki-Vote)
Benchmarks: MoleculeNet (8 datasets, ROC-AUC) TUDataset (7 datasets, accuracy)
Metrics: ROC-AUC accuracy average rank
Baselines: Vanilla Tuning StochNorm Feature Map L2-SP DELTA BSS GTOT-Tuning VGAE-Tuning Supervised
Builds on: GIN GCC structured Gromov-Wasserstein barycenters VGAE GTOT-Tuning
Closest papers by meaning
- Generative Graph Pattern Machine (NeurIPS 2025 · similarity 0.90)
- GRAVER: Generative Graph Vocabularies for Robust Graph Foundation Models Fine-tuning (NeurIPS 2025 · similarity 0.89)
- How Graph Neural Networks Learn: Lessons from Training Dynamics (ICML 2024 · similarity 0.89)
- On the Interplay between Graph Structure and Learning Algorithms in Graph Neural Networks (ICML 2025 · similarity 0.88)
- Diffusing to the Top: Boost Graph Neural Networks with Minimal Hyperparameter Tuning (ICLR 2025 · similarity 0.88)
- AdapterGNN: Parameter-Efficient Fine-Tuning Improves Generalization in GNNs (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.