Research map / Graph learning / GNN architectures
Diffusing to the Top: Boost Graph Neural Networks with Minimal Hyperparameter Tuning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GNN-Diff, a graph-conditioned latent diffusion model that generates high-performing GNN last-layer parameters from checkpoints collected under a light coarse hyperparameter search, evaluated across 166 experiments on node classification and link prediction.
Problem
Full hyperparameter tuning of GNNs is expensive in compute and time and requires careful search-space design, especially for large and long-range graphs.
Approach
Use a graph-conditioned latent DDPM (conditioned on a graph autoencoder's mean-pooled representation) to generate GNN parameters from checkpoints saved under a sub-optimal configuration chosen by a coarse random search, so the diffusion model samples parameters that can outperform grid search without extra tuning.
Key results
- GNN-Diff achieves the best average test accuracy in 36 of 40 basic node classification experiments (Cora, Citeseer, Actor, Wisconsin tables).
- On Citeseer, GCN with GNN-Diff reaches 72.37% vs 71.92% for grid search and 71.97% for coarse search.
- GNN-Diff shows lower standard deviation than baselines and generally outperforms p-diff in the graph-condition ablation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use GNN-Diff parameter generation instead of grid search | GCN · Cora node classification | accuracy: 82.04 $\pm$ 0.96 -> 82.33 $\pm$ 0.17 | Helped |
| use GNN-Diff parameter generation instead of C2F | GCN · Citeseer node classification | accuracy: 72.13 $\pm$ 0.23 -> 72.37 $\pm$ 0.29 | Helped |
| use GNN-Diff parameter generation instead of grid search | GCN · Actor node classification | accuracy: 30.94 $\pm$ 0.46 -> 31.24 $\pm$ 0.26 | Helped |
| use GNN-Diff parameter generation instead of grid search | GCN · Wisconsin node classification | accuracy: 59.80 $\pm$ 2.96 -> 61.17 $\pm$ 0.78 | Helped |
| use GNN-Diff parameter generation instead of C2F | GCN · Flickr node classification | accuracy: 52.81 $\pm$ 0.76 -> 52.84 $\pm$ 0.04 | No clear effect |
| use GNN-Diff parameter generation instead of C2F | SAGE · Reddit node classification | accuracy: 93.34 $\pm$ 0.06 -> 93.45 $\pm$ 0.03 | Helped |
| use GNN-Diff parameter generation instead of C2F | GCN · PascalVOC-SP node classification | F1-score: 23.33 $\pm$ 0.26 -> 23.52 $\pm$ 0.08 | Helped |
| use GNN-Diff parameter generation instead of grid search | SAGE · Cora link prediction | accuracy: 75.09 $\pm$ 0.49 -> 76.33 $\pm$ 0.31 | Helped |
| add the graph condition (ablation) | GNN-Diff with GCN · Cora node classification | accuracy: 81.96 $\pm$ 0.31 -> 82.33 $\pm$ 0.17 | Helped |
| add the graph condition (ablation) | GNN-Diff with SAGE · Cora node classification | accuracy: 80.19 $\pm$ 0.60 -> 80.60 $\pm$ 0.05 | Helped |
| add the graph condition (ablation) | GNN-Diff with GCN · Actor node classification | accuracy: 30.96 $\pm$ 0.30 -> 31.24 $\pm$ 0.26 | Helped |
| add the graph condition (ablation) | GNN-Diff with SAGE · Actor node classification | accuracy: 36.25 $\pm$ 0.31 -> 36.11 $\pm$ 0.09 | Hurt |
Surprising findings
- Random or coarse search results are sometimes better than full grid search on test data, which the authors attribute to overfitting the validation set with extensive grid search.
- On Chameleon link prediction, random search outperforms GNN-Diff for several target models.
Limitations
- Generation only applied to the last layer in all experiments (partial generation), so full-parameter generation is not the main setting.
- Results on some datasets (e.g., Chameleon link prediction, APPNP on OGB-Products) are not best; GNN-Diff does not always win.
- Not faster than C2F on small graphs such as Cora and Actor.
Experimental setup
Datasets: Cora Citeseer Pubmed Computers Photo CS Actor Wisconsin Roman-Empire Amazon-Ratings Minesweeper Tolokers
Benchmarks: Cora Citeseer Actor Wisconsin Flickr Reddit OGB-arXiv OGB-Products PascalVOC-SP COCO-SP Cora (link prediction) Citeseer (link prediction)
Metrics: test accuracy macro F1 (x100) standard deviation
Baselines: grid search random search coarse search coarse-to-fine search p-diff MLP
Builds on: p-diff latent diffusion models DDPM coarse-to-fine search
Closest papers by meaning
- Classic GNNs are Strong Baselines: Reassessing GNNs for Node Classification (NeurIPS 2024 · similarity 0.91)
- A Generalized Neural Diffusion Framework on Graphs (AAAI 2024 · similarity 0.90)
- Edge Prompt Tuning for Graph Neural Networks (ICLR 2025 · similarity 0.89)
- Can Classic GNNs Be Strong Baselines for Graph-level Tasks? Simple Architectures Meet Excellence (ICML 2025 · similarity 0.89)
- How Graph Neural Networks Learn: Lessons from Training Dynamics (ICML 2024 · similarity 0.89)
- Mirage: Model-agnostic Graph Distillation for Graph Classification (ICLR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.