Research map / Graph learning / GNN architectures
Forward Learning of Graph Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ForwardGNN, a forward-only (no backpropagation) layer-wise training framework for GNNs that removes negative sample construction via a single forward pass with virtual nodes or label features, and optionally adds top-down signals.
Problem
Backpropagation is the standard for training GNNs but imposes memory, locality, and parallelism constraints; existing forward-forward methods need negative samples, are designed for image classification, and do not handle node classification or link prediction on graphs.
Approach
Extends forward-forward to graphs by augmenting node features or the graph with label-based virtual nodes, then trains each GNN layer locally with a contrastive objective against class representatives produced by the layer itself in a single forward pass, optionally feeding top-down signals from previous time steps without gradient flow.
Key results
- Single-forward ForwardGNN (SF) performs on par with or better than BP on node classification and link prediction, while BP memory usage grows up to 18x with four layers and SF memory stays nearly constant.
- Top-down signals (SF-TopDown) improve node classification accuracy over SF on all five graphs, e.g., CiteSeer 91.14 to 93.58 and Amazon 89.69 to 92.09.
- FF-based methods (FF-VN, FF-LA, FF-SymBa) are often unstable and use much more memory than the single-forward approaches.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add top-down signals to SF (ablation) | SF · CiteSeer node classification | accuracy: 91.14 -> 93.58 | Helped |
| add top-down signals to SF (ablation) | SF · CoraML node classification | accuracy: 84.95 -> 86.01 | Helped |
| add top-down signals to SF (ablation) | SF · PubMed node classification | accuracy: 81.90 -> 82.21 | Helped |
| add top-down signals to SF (ablation) | SF · Amazon node classification | accuracy: 89.69 -> 92.09 | Helped |
| add top-down signals to SF (ablation) | SF · GitHub node classification | accuracy: 83.16 -> 83.68 | Helped |
| use single-forward learning instead of BP | GNNs · node classification across evaluated graphs and GNNs | accuracy | Mixed |
| use single-forward learning instead of BP | GNNs · link prediction across evaluated graphs and GNNs | ROC-AUC | Helped |
| use single-forward learning instead of FF-based methods | ForwardGNN node classification · node classification across evaluated graphs | accuracy | Mixed |
| incorporate top-down signals (ablation) | SF node classification · five graphs, averaged over GNN models and varying number of layers | accuracy | Helped |
| use ForwardGNN-CE instead of CaFo | link prediction · link prediction across evaluated graphs and GNNs | ROC-AUC | Helped |
| use ForwardGNN-CE instead of ForwardGNN-SymBa | ForwardGNN link prediction · link prediction across evaluated graphs and GNNs | ROC-AUC | Mixed |
Surprising findings
- BP link prediction performance declines as more GNN layers are added, whereas the forward-only SF method improves or stays stable with depth.
- PEPITA fails to train GNNs effectively for node classification.
Limitations
- Evaluated only on five graphs and three GNN architectures with two-layer models in the main comparisons.
- Label-based virtual node construction applies only to labeled nodes; graph classification and clustering are left as extensions without experiments.
Experimental setup
Datasets: PubMed CiteSeer CoraML Amazon GitHub
Benchmarks: PubMed CiteSeer CoraML Amazon GitHub
Metrics: classification accuracy ROC-AUC GPU memory usage
Baselines: backpropagation FF-VN FF-LA FF-SymBa CaFo PEPITA
Builds on: forward-forward algorithm CaFo SymBa PEPITA collaborative forward-forward
Closest papers by meaning
- DFA-GNN: Forward Learning of Graph Neural Networks by Direct Feedback Alignment (NeurIPS 2024 · similarity 0.91)
- Diffusing to the Top: Boost Graph Neural Networks with Minimal Hyperparameter Tuning (ICLR 2025 · similarity 0.89)
- On Vanishing Gradients, Over-Smoothing, and Over-Squashing in GNNs: Bridging Recurrent and Graph Learning (NeurIPS 2025 · similarity 0.88)
- How Graph Neural Networks Learn: Lessons from Training Dynamics (ICML 2024 · similarity 0.88)
- Can Classic GNNs Be Strong Baselines for Graph-level Tasks? Simple Architectures Meet Excellence (ICML 2025 · similarity 0.87)
- Edge Prompt Tuning for Graph Neural Networks (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.