Research map / Graph learning / GNN architectures
Efficient Graph Continual Learning via Lightweight Graph Neural Tangent Kernels-based Dataset Distillation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LIGHTGNTK, a lightweight graph neural tangent kernel dataset distillation framework that uses Bernoulli-sampled low-rank Laplacian approximation and subgraph anchoring to distill graphs for graph continual learning across node, edge, and graph-level tasks.
Problem
Fine-tuning large graph models on large noisy downstream datasets is computationally expensive, and existing graph dataset distillation methods based on GNTK are costly and limited to graph-level tasks.
Approach
Approximate the graph Laplacian with a low-rank Bernoulli-sampled eigen-decomposition inside the GNTK to cut cost to roughly O(nrd), then select training graphs by minimum gradient similarity to a validation set, with subgraph anchoring unifying node, edge, and graph tasks.
Key results
- On NCI1 with 10 graphs per class, LIGHTGNTK reaches 80.0% accuracy versus 64.8% for GNTK and 61.7% for KIDD (as reported in Table 1).
- LIGHTGNTK reduces distillation computation time versus GNTK across all graph datasets, e.g., NCI1 from 196s to 144s and ogbg-molhiv from 921s to 683s.
- Using Bernoulli sampling with sampling rate 0.1 is reported as a favorable performance-efficiency trade-off; Bernoulli sampling outperforms top-r and bottom-r eigenvalue selection.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LIGHTGNTK instead of GNTK | Graph dataset distillation with GIN · NCI1 (Graphs/Cl=10) | ACC: 64.8±0.2 -> 64.4±3.3 | No clear effect |
| Use LIGHTGNTK instead of KIDD | Graph dataset distillation with GIN · NCI109 (Graphs/Cl=10) | ACC: 63.5±0.5 -> 64.2±4.3 | Helped |
| Use LIGHTGNTK instead of KIDD | Graph dataset distillation with GIN · PROTEINS (Graphs/Cl=10) | ACC: 74.1±1.9 -> 74.7±5.5 | No clear effect |
| Use LIGHTGNTK instead of KIDD | Graph dataset distillation with GIN · DD (Graphs/Cl=10) | ACC: 70.6±1.3 -> 71.5±1.7 | Helped |
| Use LIGHTGNTK instead of GNTK | Graph dataset distillation with GIN · ogbg-molhiv (Graphs/Cl=10) | ROC-AUC: 69.2±0.2 -> 68.5±8.7 | No clear effect |
| Use LIGHTGNTK instead of KIDD | Graph dataset distillation with GIN · ogbg-molbbbp (Graphs/Cl=10) | ROC-AUC: 64.4±1.5 -> 64.6±6.1 | No clear effect |
| Use LIGHTGNTK instead of KIDD | Graph dataset distillation with GIN · ogbg-molbace (Graphs/Cl=10) | ROC-AUC: 74.8±2.0 -> 70.8±9.6 | Hurt |
| Use LIGHTGNTK instead of Herding | Node dataset distillation with GIN · CiteSeer (Nodes/Cl=10) | accuracy: 63.7±0.9 -> 62.7±5.1 | Hurt |
| Use LIGHTGNTK instead of Herding | Node dataset distillation with GIN · Cora (Nodes/Cl=10) | accuracy: 75.7±1.8 -> 76.2±0.6 | Helped |
| Use LIGHTGNTK instead of GNTK | Node dataset distillation with GIN · PubMed (Nodes/Cl=10) | accuracy: 70.1±2.0 -> 70.7±2.1 | No clear effect |
| Use LIGHTGNTK instead of GNTK | Node dataset distillation with GIN · ogbn-arxiv (Nodes/Cl=10) | accuracy: 18.2±0.8 -> 18.9±3.2 | Helped |
| Use Bernoulli eigenvalue sampling instead of top-r or bottom-r eigenvalue selection (ablation) | LIGHTGNTK · Graph dataset distillation benchmarks | performance | Helped |
Surprising findings
- Reported speedups are modest (about 6% to 27%) rather than order-of-magnitude, despite the claimed O(nrd) complexity versus O(n^2 d).
Limitations
- Speedup over GNTK is modest on several datasets (e.g., PROTEINS 26.3s to 24.8s).
- Performance is inconsistent: several baselines (e.g., KIDD, GNTK) match or beat LIGHTGNTK on some settings, and standard deviations are large.
Experimental setup
Datasets: NCI1 NCI109 PROTEINS DD ogbg-molhiv ogbg-molbbbp ogbg-molbace Cora CiteSeer PubMed ogbn-arxiv ogbl-collab
Benchmarks: NCI1 NCI109 PROTEINS DD ogbg-molhiv ogbg-molbbbp ogbg-molbace Cora CiteSeer PubMed ogbn-arxiv ogbl-collab
Metrics: accuracy ROC-AUC computation time
Baselines: Random Sampling Herding K-Center Greedy DosCond KIDD GNTK
Builds on: GNTK KIDD GCOND GPT-GNN
Closest papers by meaning
- Temporal Graph Neural Tangent Kernel with Graphon-Guaranteed (NeurIPS 2024 · similarity 0.89)
- A Topology-aware Graph Coarsening Framework for Continual Graph Learning (NeurIPS 2024 · similarity 0.88)
- How Graph Neural Networks Learn: Lessons from Training Dynamics (ICML 2024 · similarity 0.88)
- Dynamic Sub-graph Distillation for Robust Semi-supervised Continual Learning (AAAI 2024 · similarity 0.87)
- Exploring Rationale Learning for Continual Graph Learning (AAAI 2025 · similarity 0.87)
- A Selective Learning Method for Temporal Graph Continual Learning (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.