Research map / Graph learning / Knowledge and temporal graphs
Simple and Efficient Heterogeneous Temporal Graph Neural Network
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SE-HTGNN, a heterogeneous temporal GNN that unifies spatial and temporal learning via a GRU-based dynamic attention over historical attention coefficients, initialized with LLaMA3-8B type-prompt embeddings, reporting up to 10x training speedup.
Problem
Heterogeneous temporal graph (HTG) forecasting with existing heterogeneous dynamic GNNs that decouple spatial and temporal modeling, which weakens spatio-temporal interaction and yields high model complexity and slow training.
Approach
Integrates temporal modeling into spatial attention: relation-wise GRUs carry historical attention coefficients as hidden states to guide attention on subsequent snapshots, uses non-parametric GCN aggregation, and initializes the GRU hidden state from LLM-derived node-type embeddings.
Key results
- SE-HTGNN achieves AUC 93.13% on OGBN-MAG and 91.08% on Aminer, outperforming the best baseline CasMLN (90.85%, 88.53%) and HTGNN.
- On COVID-19 30-day regression, SE-HTGNN reaches MAE 497 vs 536 for the next best DHGAS, and up to ~7% MAE improvement on 60/90-day horizons.
- Training cost on COVID-19 drops to 64 GPU seconds versus 637 for CasMLN and 11,120 for DHGAS, reported as up to ~10x speed-up.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SE-HTGNN instead of the strongest reported baseline | heterogeneous temporal graph forecasting · OGBN-MAG and Aminer link prediction; YELP node classification | AUC, AP, Macro-F1, and Recall: +2.11%; +3.62%; +2.89%; +3.19%; +4.81%; +4.96% | Helped |
| Use SE-HTGNN instead of the strongest reported baseline | COVID-19 forecasting · COVID-19 30-day prediction | MAE: +7.27% | Helped |
| Use SE-HTGNN instead of the strongest reported baseline | COVID-19 forecasting · COVID-19 60-day and 90-day prediction | MAE: +8.44%; +6.97% | Helped |
| Use SE-HTGNN instead of CasMLN | HTGNN training · OGBN-MAG, Aminer, YELP, and COVID-19 | GPU seconds: 2.7× speedup; nearly a 10× speedup | Helped |
| Replace LLM-based initialization with random initialization (ablation) | SE-HTGNN · OGBN-MAG link prediction | AUC (%): 93.13 ± 0.56 -> 90.87 ± 1.24 | Hurt |
| Replace LLM-based initialization with average initialization (ablation) | SE-HTGNN · Aminer link prediction | AUC (%): 91.08 ± 0.59 -> 89.76 ± 0.34 | Hurt |
| Replace LLM-based initialization with zero initialization (ablation) | SE-HTGNN · OGBN-MAG link prediction | AUC (%): 93.13 ± 0.56 -> 91.78 ± 0.68 | Hurt |
| Replace dynamic attention with projected attention (ablation) | SE-HTGNN · OGBN-MAG link prediction | AUC (%): 93.13 ± 0.56 -> 86.83 ± 1.21 | Hurt |
| Replace dynamic attention with self-attention (ablation) | SE-HTGNN · OGBN-MAG link prediction | AUC (%): 93.13 ± 0.56 -> 91.65 ± 1.22 | Hurt |
| Replace dynamic attention with gated attention (ablation) | SE-HTGNN · OGBN-MAG link prediction | AUC (%): 93.13 ± 0.56 -> 87.94 ± 2.42 | Hurt |
| Replace simplified neighbor aggregation with no aggregation (ablation) | SE-HTGNN · OGBN-MAG link prediction | AUC (%): 93.13 ± 0.56 -> 83.91 ± 1.20 | Hurt |
| Replace simplified neighbor aggregation with GCN aggregation (ablation) | SE-HTGNN · OGBN-MAG link prediction | AUC (%): 93.13 ± 0.56 -> 90.57 ± 0.86 | Hurt |
Surprising findings
- Node-level attention is found to be trivial on HTGs while relation-level attention is vital, per an analysis on HTGNN and DyHATR.
- Random initialization of the GRU hidden state degrades performance most among LLM-initialization alternatives on OGBN-MAG (90.87 vs 93.13 AUC).
Limitations
- LLM embeddings have high fixed dimensionality, so linear projection adds parameter overhead that can reduce efficiency.
- Dimensionality reduction (low-rank, PCA) of LLM outputs is left to future work.
- Reported improvements are averaged over only 5 runs and dataset coverage is limited to four datasets.
Experimental setup
Datasets: OGBN-MAG Aminer YELP COVID-19
Benchmarks: OGBN-MAG Aminer YELP COVID-19
Metrics: AUC AP Macro-F1 Recall MAE RMSE GPU training time
Baselines: GCN GAT TGAT RGCN RGAT HGT DiffMG DyHATR HGT+ HTGNN DHGAS CasMLN
Builds on: HTGNN DyHATR GAT HAN GCN
Closest papers by meaning
- Temporal-Aware Evaluation and Learning for Temporal Graph Neural Networks (AAAI 2025 · similarity 0.88)
- PRES: Toward Scalable Memory-Based Dynamic Graph Neural Networks (ICLR 2024 · similarity 0.88)
- State Space Models on Temporal Graphs: A First-Principles Study (NeurIPS 2024 · similarity 0.87)
- An Efficient Subgraph-Inferring Framework for Large-Scale Heterogeneous Graphs (AAAI 2024 · similarity 0.87)
- HGOT: Self-supervised Heterogeneous Graph Neural Network with Optimal Transport (ICML 2025 · similarity 0.87)
- Towards Dynamic Spatial-Temporal Graph Learning: A Decoupled Perspective (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.