Research map / Graph learning / Knowledge and temporal graphs

PRES: Toward Scalable Memory-Based Dynamic Graph Neural Networks

ICLR 2024 · Graph learning › Knowledge and temporal graphs · approach: Temporal graph neural architectures · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes PRES, a prediction-correction scheme with a memory coherence smoothing loss that lets memory-based dynamic GNNs (TGN, JODIE, APAN) train with ~4x larger temporal batches and up to 3.4x speed-up with little accuracy loss.

Problem

Training memory-based dynamic graph neural networks with large temporal batches causes temporal discontinuity (intra-batch pending events are processed in parallel), which degrades performance and limits data parallelism.

Approach

Treat memory states affected by pending events in a batch as noisy measurements; predict the memory update with a Gaussian mixture model over memory changes and fuse the prediction with the measured state via a learnable gain, plus a memory coherence regularization term that encourages gradient alignment despite pending events.

Key results

• TGN-PRES reaches 98.0% AP on REDDIT versus 98.4% for TGN with a 3.4x training speed-up per epoch.
• PRES enables up to 4x larger temporal batch size while maintaining AP across JODIE, TGN, and APAN on five datasets.
• Table 2 shows node classification ROC-AUC changes within roughly 1-2 points for PRES variants relative to baselines (e.g., TGN WIKI 86.8 vs 84.7).

Measured effects reported in the paper

Surprising findings

• Smaller temporal batch sizes can yield larger gradient variance and poorer or non-converging performance, contrary to the common belief that small batches are always better.
• Theorem 1 shows the variance of the full-epoch gradient under negative sampling is lower-bounded by |E|/b times sigma_min^2, so larger temporal batches are more robust to sampling noise.

Limitations

• Analysis and method restricted to the MDGNN family; extension to time series is left for future work.
• Theoretical guarantees rely on assumptions (bounded variance, Lipschitz gradients, lower-bounded memory coherence) and a linear state-space approximation for the variance-reduction proposition.
• Some datasets show small AP drops with PRES (e.g., LASTFM TGN 72.1 to 71.2, GDELT 96.8 to 96.0).

Experimental setup

Datasets: WIKI REDDIT MOOC LASTFM GDELT

Benchmarks: WIKI REDDIT MOOC LASTFM GDELT

Metrics: average precision ROC-AUC speedup

Baselines: TGN JODIE APAN

Builds on: TGN JODIE APAN TGL

Closest papers by meaning

• Temporal-Aware Evaluation and Learning for Temporal Graph Neural Networks (AAAI 2025 · similarity 0.88)
• Simple and Efficient Heterogeneous Temporal Graph Neural Network (NeurIPS 2025 · similarity 0.88)
• State Space Models on Temporal Graphs: A First-Principles Study (NeurIPS 2024 · similarity 0.87)
• Delay-DSGN: A Dynamic Spiking Graph Neural Network with Delay Mechanisms for Evolving Graph (ICML 2025 · similarity 0.87)
• Node-Time Conditional Prompt Learning in Dynamic Graphs (ICLR 2025 · similarity 0.87)
• Using Time-Aware Graph Neural Networks to Predict Temporal Centralities in Dynamic Graphs (NeurIPS 2024 · similarity 0.86)