Research map / Graph learning / GNN architectures
Module-Aware Optimization for Graph Neural Networks
Guy Hadad, Haggai Roitman, Moshe Eliasof, Bracha Shapira
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a module-aware optimizer for GNNs that treats channel-mixing weights and graph-propagation components differently, rather than applying one update geometry to all parameters.
Problem
Optimizers such as Adam/AdamW (coordinate-wise) and Muon (orthogonalized matrix updates) impose fixed geometries on parameter updates, but in GNNs the effective geometry of weight matrices is also shaped by graph propagation, which these optimizers ignore.
Approach
Makes the optimizer aware of GNN module structure: updates are chosen per module, accounting for the graph-propagation operator as well as learned channel-mixing weights, instead of applying one geometry uniformly.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add graph-aware orthogonalization and step normalization to Muon | Muon · ogbn-arxiv, ogbn-mag, ogbn-products (GCN) | validation-loss time-to-target: 1.32x-2.42x speedup over AdamW | Improved |
| add graph-aware orthogonalization alone to Muon (ablation) | Muon · ogbn-arxiv | speedup vs Muon: 1.00x -> 5.39x | Improved |
| add graph-aware orthogonalization alone to Muon (ablation) | Muon · ogbn-mag, ogbn-products | speedup vs Muon: 1.07x and 1.03x | No clear effect |
| add graph-aware step normalization alone to Muon (ablation) | Muon · ogbn-mag | speedup vs Muon: 1.00x -> 2.28x | Improved |
| combine graph-aware orthogonalization with step normalization (ablation) | GraphMuon · ogbn-mag | speedup vs Muon: 2.28x -> 3.60x | Improved |
| use GraphMuon instead of AdamW | GAT · ogbn-arxiv | test accuracy: .6928 -> .6926 | No clear effect |
| use GraphMuon instead of AdamW | GraphGPS · ogbn-mag | test accuracy: .2939 -> .3296 | Improved |
| use GraphMuon instead of AdamW | GINE · ZINC | MAE: .2303 -> .1897 | Improved |
| use GraphMuon instead of AdamW | GINE · ogbg-moltox21 | wall-clock speedup: 1.20x speedup, ROC-AUC .6633 -> .6644 | Mixed |
| use Muon instead of AdamW | Muon · ogbn-products, ogbg-molbace | time-to-target: did not reach target | Hurt |
Experimental setup
Baselines: Adam AdamW Muon
Builds on: Muon Adam AdamW
Closest papers by meaning
- Higher-Order Graphon Neural Networks: Approximation and Cut Distance (ICLR 2025 · similarity 0.86)
- From SGD to Muon: Adaptive Optimization via Schatten-p Norms (NeurIPS 2026 · similarity 0.86)
- Towards Understanding the Power and Limits of the Muon Optimizer: A River-Valley Perspective (NeurIPS 2026 · similarity 0.86)
- A Few Moments Please: Scalable Graphon Learning via Moment Matching (NeurIPS 2025 · similarity 0.85)
- How Graph Neural Networks Learn: Lessons from Training Dynamics (ICML 2024 · similarity 0.85)
- Tensorion: A Tensor-Aware Generalization of the Muon Optimizer (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
