Research map / Graph learning / GNN architectures
Neural Tangent Kernels Motivate Cross-Covariance Graphs in Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Uses NTK analysis of graph neural networks to show that alignment (yᵀΘy) is maximized by using the input-output cross-covariance as the graph shift operator, and validates this on HCP-YA rfMRI time-series prediction.
Problem
Choosing the graph shift operator for GNNs on multivariate regression is usually based only on input data covariance; it is unclear what graph best supports fast convergence and generalization for supervised prediction.
Approach
Defines alignment A = yᵀΘy between the NTK and outputs, which governs gradient descent convergence and generalization; shows that for graph filters and two-layer tanh GNNs, optimizing a lower bound on alignment yields a GSO proportional to the cross-covariance (XYᵀ+YXᵀ)/2 minus identity-type terms.
Key results
- Cross-covariance GSO (CXY) yields faster training-loss convergence and lower training error than covariance GSO (CXX) for graph filters and GNNs on HCP-YA.
- Test-error advantage of CXY over CXX holds consistently across prediction horizons Δt in {1,...,5} averaged over individuals.
- Cross-covariance GNNs trained on 50 features transfer to 100-feature data with a relative loss gain similar to the 100-feature baseline setting.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace the input covariance graph CXX with the input-output cross-covariance graph CXY | GNNs and graph filters · HCP-YA time-series prediction | training loss | Helped |
| replace the input covariance graph CXX with the input-output cross-covariance graph CXY | GNNs and graph filters · HCP-YA time-series prediction across ∆t ∈ {1, 2, 3, 4, 5} | test error | Helped |
| replace the input covariance graph CXX with the input-output cross-covariance graph CXY | GNNs · HCP-YA time-series prediction | training error | Helped |
| replace the input covariance graph CXX with the input-output cross-covariance graph CXY | transferred GNNs · HCP-YA time-series prediction, trained on 50 features and tested on 100 features | test loss | Helped |
Limitations
- Theory restricted to inputs and outputs of the same dimensionality
- Tightness of the lower bounds on alignment not analyzed
- Theoretical results only for two-layer GNN with tanh; ReLU and deeper cases supported only empirically
Experimental setup
Datasets: HCP-YA rfMRI
Benchmarks: HCP-YA rfMRI time series prediction
Metrics: training loss test error test loss gap between CXY and CXX
Baselines: covariance graph GNN (CXX GSO, VNN) graph filter with covariance GSO
Builds on: neural tangent kernel graph filters (Gama et al., 2020) vector neural networks (Sihag et al., 2022) Arora et al. 2019 gradient descent convergence NTK
Closest papers by meaning
- How Graph Neural Networks Learn: Lessons from Training Dynamics (ICML 2024 · similarity 0.90)
- Temporal Graph Neural Tangent Kernel with Graphon-Guaranteed (NeurIPS 2024 · similarity 0.89)
- Equivariant Neural Tangent Kernels (ICML 2025 · similarity 0.87)
- Exploring Consistency in Graph Representations: from Graph Kernels to Graph Neural Networks (NeurIPS 2024 · similarity 0.87)
- Understanding the Evolution of the Neural Tangent Kernel at the Edge of Stability (NeurIPS 2025 · similarity 0.87)
- The Surprising Effectiveness of Infinite-Width NTKs for Characterizing and Improving Model Training (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.