Research map / Graph learning / Knowledge and temporal graphs

Temporal-Aware Evaluation and Learning for Temporal Graph Neural Networks

AAAI 2025 · Graph learning › Knowledge and temporal graphs · approach: Temporal graph neural architectures · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Volatility-Cluster Statistics (VCS), an evaluation metric for TGNN errors based on Hopkins statistics, and a differentiable VCA training regularizer that reduces error volatility clustering across six TGNNs on five datasets.

Problem

Instance-based metrics such as AP and AU-ROC treat each test event independently, so they cannot distinguish TGNNs whose errors cluster in time (volatility clusters), which matters for model selection and fault-tolerant systems.

Approach

Formally show instance-based metrics reduce predictions to a disagreement count, then measure the temporal clustering of disagreement events relative to random sampling via a Hopkins-style distance ratio (VCS), and turn a smooth version of it into a regularizer (VCA) added to the BCE loss.

Key results

• VCA lowers VCS for every evaluated TGNN; e.g., TGN on Reddit goes from 0.18 to 0.08 and the average VCS reduction is about 0.09-0.1 per dataset.
• Memory-based (TGN, Tiger) and RNN-based (JOIDE, DyRep) models show larger VCS than attention-based TCL and TGAT, which are more uniform.
• The gain in VCS comes with small AP changes; e.g., TGN on LastFM drops from 75.4 to 73.3 AP while Tiger on LastFM rises from 77.7 to 78.0.

Measured effects reported in the paper

Surprising findings

• Instance-based metrics such as AP give nearly identical scores across TGNN architectures, so they cannot separate models by error pattern.
• Memory-based TGNNs show errors clustering at the end of the test period, while RNN-based TGNNs show them at the beginning.

Limitations

• VCS captures only volatility clustering and not the arrival time of errors.
• Increasing the VCA weight gamma improves VCS but lowers AP, creating a trade-off.
• The study is limited to link prediction and to temporal rather than spatial structure.

Experimental setup

Datasets: Reddit Wikipedia MOOC LastFM GDELT

Benchmarks: Reddit Wikipedia MOOC LastFM GDELT

Metrics: VCS AP AU-ROC

Baselines: TGN Tiger TCL TGAT JOIDE DyRep

Builds on: Hopkins statistics TGN TGAT TCL DyRep

Closest papers by meaning

• Simple and Efficient Heterogeneous Temporal Graph Neural Network (NeurIPS 2025 · similarity 0.88)
• Using Time-Aware Graph Neural Networks to Predict Temporal Centralities in Dynamic Graphs (NeurIPS 2024 · similarity 0.88)
• PRES: Toward Scalable Memory-Based Dynamic Graph Neural Networks (ICLR 2024 · similarity 0.88)
• State Space Models on Temporal Graphs: A First-Principles Study (NeurIPS 2024 · similarity 0.87)
• TGB 2.0: A Benchmark for Learning on Temporal Knowledge Graphs and Heterogeneous Graphs (NeurIPS 2024 · similarity 0.86)
• TGB-Seq Benchmark: Challenging Temporal GNNs with Complex Sequential Dynamics (ICLR 2025 · similarity 0.86)