Research map / Graph learning / GNN architectures
Bundle Neural Network for message diffusion on graphs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Bundle Neural Networks (BuNNs), which perform global message diffusion via the heat kernel on learned flat vector bundles, showing they avoid over-smoothing and over-squashing, are compact uniform universal approximators given injective positional encodings, and achieve strong heterophilic and LRGB results.
Problem
Message passing GNNs suffer from over-smoothing, over-squashing, and limited expressivity; existing sheaf neural networks remain local message passing models with costly bundle learning.
Approach
Derive a GNN layer from heat diffusion over flat vector bundles, where orthogonal node maps factor the bundle Laplacian so the heat kernel reduces to the standard graph heat kernel times learned orthogonal maps, enabling global, non-message-passing diffusion that avoids over-smoothing.
Key results
- BuNN reaches 91.75% accuracy on roman-empire versus 88.75% for the best listed baseline (GAT-sep), and 98.99 ROC AUC on minesweeper.
- On Peptides-func BuNN achieves 72.76 test AP, a new state of the art over GAPH ViT's 69.42 in the reported table.
- On the synthetic barbell over-squashing task BuNN reaches MSE 0.01 while GCN, SAGE, GAT, GPS and NSD stay around 0.9 to 1.1.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace SAGE with BuNN | SAGE · Barbell (over-squashing) | MSE: $0.90\pm 0.29$ -> $\mathbf{0.01\pm 0.07}$ | Helped |
| Replace NSD with BuNN | NSD · Clique (over-smoothing) | MSE: $0.08 \pm 0.02$ -> $\mathbf{0.03 \pm 0.01}$ | Helped |
| Replace GAT-sep with BuNN | GAT-sep · roman-empire | accuracy: $88.75 \pm 0.41$ -> $\mathbf{91.75 \pm 0.39}$ | Helped |
| Replace SAGE with BuNN | SAGE · amazon-ratings | accuracy: $53.63 \pm 0.39$ -> $\mathbf{53.74\pm 0.51}$ | Helped |
| Replace GAT-sep with BuNN | GAT-sep · minesweeper | ROC AUC: $93.91 \pm 0.35$ -> $\mathbf{98.99\pm 0.16}$ | Helped |
| Replace GAT-sep with BuNN | GAT-sep · tolokers | ROC AUC: $83.78 \pm 0.43$ -> $\mathbf{84.78\pm 0.80}$ | Helped |
| Replace GT-sep with BuNN | GT-sep · questions | ROC AUC: $78.05 \pm 0.93$ -> $\mathbf{78.75\pm 1.09}$ | Helped |
| Replace DReW with BuNN | DReW · Peptides-func | Test AP: $71.50\pm0.44$ -> $\mathbf{72.76\pm 0.65}$ | Helped |
| Replace GAPH ViT with BuNN | GAPH ViT · Peptides-struct | Test MAE: $\mathbf{24.49}\pm0.16$ -> $24.63\pm 0.12$ | Hurt |
| Replace GPS with BuNN | GPS · PascalVOC-SP | Test F1: $\mathbf{44.40\pm 0.65}$ -> $ 40.49\pm 0.46 $ | Hurt |
Surprising findings
- GPS (fully connected graph transformer) fails to generalize on the clique over-smoothing task, with MSE equal to the constant baseline.
- NSD performs far below BuNN on heterophilic tasks, which the authors attribute to NSD learning a restriction map per node-edge pair and overfitting.
Limitations
- Heat kernel for large diffusion time requires spectral methods, which are computationally expensive.
- Theory assumes injective node-level positional encodings, which may not always be available.
- No link prediction experiments.
Experimental setup
Datasets: roman-empire amazon-ratings minesweeper tolokers questions Peptides-func Peptides-struct PascalVOC-SP synthetic barbell and clique tasks
Benchmarks: roman-empire amazon-ratings minesweeper tolokers questions Peptides-func Peptides-struct PascalVOC-SP
Metrics: accuracy ROC AUC test AP test MAE test F1 mean squared error
Baselines: GCN GraphSAGE GAT NSD GraphGPS MLP GAT-sep GT GT-sep GINE GatedGCN DReW
Builds on: Neural Sheaf Diffusion GCN graph transformers Householder reflections
Closest papers by meaning
- Mitigating Over-squashing without Rewiring: A Sheaf Effective Resistance Perspective (NeurIPS 2026 · similarity 0.88)
- Cooperative Graph Neural Networks (ICML 2024 · similarity 0.87)
- What do Graph Neural Networks learn? Insights from Tropical Geometry (NeurIPS 2024 · similarity 0.87)
- Improving Graph Neural Networks by Learning Continuous Edge Directions (ICLR 2025 · similarity 0.87)
- Dynamic Bundling with Large Language Models for Zero-Shot Inference on Text-Attributed Graphs (NeurIPS 2025 · similarity 0.87)
- Towards Dynamic Message Passing on Graphs (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.