Research map / Graph learning / GNN architectures
Simple Orthogonal Graph Representation Learning (Student Abstract)
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PDOI-R, a polar-decomposition-based orthogonal initialization (via randomized SVD) for GNN weights, to stabilize forward/backward signal propagation and improve downstream GNN performance.
Problem
GNNs become hard to train as depth grows because stacking layers destabilizes forward activation and backward gradient propagation; adapting large GNNs to downstream tasks efficiently is also a concern.
Approach
Impose approximate orthogonality on GNN weight matrices by applying an orthogonal initialization derived from the polar decomposition of the Euclidean gradient (computed with randomized SVD), mapping parameters onto the Stiefel manifold to preserve norms and stabilize training.
Key results
- PDOIR-GraphMAE improves over GraphMAE on molecular property benchmarks, e.g., ClinTox 82.3 to 86.1 and MUV 76.3 to 79.2 ROC-AUC.
- PDOIR-GCNII reaches 90.95% on Cora vs 88.01% for GCNII in full-supervised node classification.
- Applying PDOI-R initialization to EGNN lowers validation loss curves on ISO17 compared with Xavier initialization (figure only).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add PDOI-R orthogonal initialization to GraphMAE | GraphMAE · BBBP | ROC-AUC (%): 72.5±0.8 -> 73.0±0.5 | Helped |
| Add PDOI-R orthogonal initialization to GraphMAE | GraphMAE · Tox21 | ROC-AUC (%): 75.7±0.5 -> 77.25±0.4 | Helped |
| Add PDOI-R orthogonal initialization to GraphMAE | GraphMAE · ToxCast | ROC-AUC (%): 64.1±0.3 -> 65.3±0.4 | Helped |
| Add PDOI-R orthogonal initialization to GraphMAE | GraphMAE · SIDER | ROC-AUC (%): 60.3±1.1 -> 64.7±1.0 | Helped |
| Add PDOI-R orthogonal initialization to GraphMAE | GraphMAE · ClinTox | ROC-AUC (%): 82.3±1.2 -> 86.1±1.2 | Helped |
| Add PDOI-R orthogonal initialization to GraphMAE | GraphMAE · MUV | ROC-AUC (%): 76.3±2.4 -> 79.2±1.8 | Helped |
| Add PDOI-R orthogonal initialization to GraphMAE | GraphMAE · HIV | ROC-AUC (%): 78.5±1.2 -> 79.9±0.7 | Helped |
| Add PDOI-R orthogonal initialization to GraphMAE | GraphMAE · BACE | ROC-AUC (%): 83.5±1.2 -> 85.0±0.7 | Helped |
| Add PDOI-R orthogonal initialization to GCNII | GCNII · Cora | accuracy (%): 88.69 -> 90.95 | Helped |
| Add PDOI-R orthogonal initialization to GCNII | GCNII · Cite. | accuracy (%): 77.21 -> 79.58 | Helped |
| Add PDOI-R orthogonal initialization to GCNII | GCNII · Pumb. | accuracy (%): 90.42 -> 90.59 | Helped |
| Add PDOI-R orthogonal initialization to GCNII | GCNII · Corn. | accuracy (%): 77.84 -> 78.38 | Helped |
Limitations
- Evidence is limited to short abstract-length experiments; the reported gains are compared against single baselines per setting.
- Theoretical justification for the low intrinsic dimension claim is asserted rather than rigorously derived.
Experimental setup
Datasets: ZINC15 ISO17 Cora Citeseer Pubmed Cornell Texas Wisconsin BBBP Tox21 ToxCast SIDER
Benchmarks: MoleculeNet (BBBP, Tox21, ToxCast, SIDER, ClinTox, MUV, HIV, BACE) Cora Citeseer Pubmed Cornell Texas Wisconsin ISO17
Metrics: ROC-AUC accuracy mean absolute error
Baselines: GraphMAE GraphCL GraphLoG GCNII Ortho-GCNII
Builds on: GCNII GraphMAE Ortho-GCNII EGNN
Closest papers by meaning
- Orthogonal Graph Neural Networks (AAAI 2022 · similarity 0.90)
- Adaptive Kernel Graph Neural Network (AAAI 2022 · similarity 0.88)
- How Graph Neural Networks Learn: Lessons from Training Dynamics (ICML 2024 · similarity 0.87)
- Fast Learning of Graph Neural Networks with Guaranteed Generalizability: One-hidden-layer Case (ICML 2020 · similarity 0.87)
- Exploring Consistency in Graph Representations: from Graph Kernels to Graph Neural Networks (NeurIPS 2024 · similarity 0.86)
- Generalizing CNNs to graphs with learnable neighborhood quantization (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.