Research map / Graph learning / GNN architectures

Distilling Graph Geometry: Knowledge Gap from GNNs to MLPs

NeurIPS 2026 · Graph learning › GNN architectures · approach: GNN generalization theory

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes G²MLP, a GNN-to-MLP distillation method that uses Ollivier–Ricci curvature to allocate prediction-level (KL) and representation-level (diagonal Gaussian W2) alignment, keeping a graph-free MLP at inference.

Problem

GNN-to-MLP distillation transfers node-wise predictions uniformly and does not specify where the student should preserve the teacher's graph-induced representation geometry, leading to spectral underfit on sparse graphs and spectral overfit on dense graphs.

Approach

Uses curvature-dependent opposite weights to stratify logit-side and feature-side Gaussian W2 neighborhood alignment between teacher and student, with a closed-form diagonal surrogate, while the deployed model stays a vanilla MLP.

Key results

• On transductive node classification, G²MLP reaches 74.51% on Citeseer, 78.17% on Pubmed, 82.54% on Cora, surpassing GLNN and the strongest baselines on all six datasets.
• On ogbn-arxiv transductive, G²MLP achieves 64.57% vs GLNN 63.75% and KRD 59.26%, but remains below the SAGE teacher at 70.73%.
• On link prediction (multi-task), G²MLP achieves 83.90 AUC on Cora, beating KRD's 79.20.

Surprising findings

• Plain MLP exceeds the SAGE teacher on Pubmed link prediction (90.29 vs 88.36 AUC).
• MLP student meets or exceeds its transformer teacher on 13 of 15 cells.

Limitations

• Gaussian W2 diagonal surrogate may be inaccurate for multimodal or strongly heterophilic neighborhoods.
• Exact Ollivier–Ricci curvature cost scales poorly to very large graphs; approximations (Sinkhorn, Forman) are discussed but not evaluated.
• Spectral claims rest on heuristic diagnostics (rank gap) and approximate kernel analysis rather than exact spectra.

Experimental setup

Datasets: Cora Citeseer Pubmed A-computer A-photo ogbn-arxiv

Benchmarks: Cora Citeseer Pubmed A-computer A-photo ogbn-arxiv

Metrics: accuracy AUC AP effective rank gap production score

Baselines: MLP GLNN KRD FF-G2M

Builds on: GLNN WKD KRD FF-G2M GraphSAGE

Closest papers by meaning

• VQGraph: Rethinking Graph Representation Space for Bridging GNNs and MLPs (ICLR 2024 · similarity 0.90)
• TINED: GNNs-to-MLPs by Teacher Injection and Dirichlet Energy Distillation (ICML 2025 · similarity 0.89)
• Geometry Awakening: Cross-Geometry Learning Exhibits Superiority over Individual Structures (NeurIPS 2024 · similarity 0.89)
• Neural Snowflakes: Universal Latent Graph Inference via Trainable Latent Geometries (ICLR 2024 · similarity 0.88)
• What do Graph Neural Networks learn? Insights from Tropical Geometry (NeurIPS 2024 · similarity 0.88)
• Geometry-Aware Edge Pooling for Graph Neural Networks (NeurIPS 2025 · similarity 0.88)