Research map / Graph learning / Knowledge and temporal graphs
TGB-Seq Benchmark: Challenging Temporal GNNs with Complex Sequential Dynamics
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces TGB-Seq, an eight-dataset future link prediction benchmark with low repeat edges, showing existing temporal GNNs degrade sharply versus a sequential recommender (SGNN-HN).
Problem
Existing temporal graph benchmarks contain many repeated edges and lack complex sequential dynamics, so temporal GNNs look strong there but fail on recommendation-style future link prediction involving unseen edges.
Approach
A benchmark of eight large temporal graph datasets curated to minimize repeated edges across recommendation, Who-To-Follow, citation, and web link domains, evaluated with MRR over 100 random negatives, plus a toy sequential-dynamics test showing temporal GNNs stay near 50% AP.
Key results
- On the toy sequential-dynamics dataset, all nine temporal GNNs score about 50% AP (CAWN with length 2 reaches 52.80%), while SGNN-HN reaches 100%.
- On recommendation datasets, SGNN-HN achieves MRR of 33.12 on ML-20M, 68.58 on Taobao and 62.88 on GoogleLocal, versus at most 23.99, 60.28 and 54.13 for temporal GNNs.
- Method rankings vary widely across TGB-Seq datasets; JODIE and TGN perform best on Patent and WikiLink, while DyGFormer leads on Flickr and GraphMixer on YouTube.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SGNN-HN instead of temporal GNNs | ML-20M future link prediction · ML-20M | MRR: 21.16 ± 0.73 -> 33.12 ± 0.01 | Helped |
| Use SGNN-HN instead of temporal GNNs | Taobao future link prediction · Taobao | MRR: 60.28 ± 0.54 -> 68.58 ± 0.21 | Helped |
| Add node features (ablation) | DyRep on GoogleLocal · GoogleLocal | MRR: 37.73 -> 47.41 | Helped |
| Add node features (ablation) | TGAT on GoogleLocal · GoogleLocal | MRR: 19.78 -> 30.50 | Helped |
| Add node features (ablation) | TGN on GoogleLocal · GoogleLocal | MRR: 54.13 -> 60.34 | Helped |
| Add edge features (ablation) | JODIE on GoogleLocal · GoogleLocal | MRR: 41.86 -> 42.44 | Helped |
| Add edge features (ablation) | DyRep on GoogleLocal · GoogleLocal | MRR: 37.73 -> 37.22 | Hurt |
| Add edge features (ablation) | TGAT on GoogleLocal · GoogleLocal | MRR: 19.78 -> 17.35 | Hurt |
| Add edge features (ablation) | TGN on GoogleLocal · GoogleLocal | MRR: 54.13 -> 53.48 | Hurt |
| Add node features (ablation) | CAWN on GoogleLocal · GoogleLocal | MRR: 18.26 -> 15.66 | Hurt |
| Add edge features (ablation) | TCL on GoogleLocal · GoogleLocal | MRR: 18.30 -> 8.58 | Hurt |
| Increase temporal random walk length from 1 to 2 (ablation) | CAWN on the toy sequential dynamics dataset · Toy sequential dynamics dataset | AP (%): 50.00 ± 0.00 -> 52.80 ± 0.05 | Helped |
Surprising findings
- Temporal GNNs stay near chance (about 50% AP) on a simple toy sequential pattern that a sequential recommender solves perfectly.
- Several temporal GNNs (e.g., DyGFormer, TGAT, CAWN) run out of time (24h per epoch) on large TGB-Seq datasets.
Limitations
- Datasets exclude node and edge features, so feature-based methods are not fully tested on the main benchmark.
- Most TGB-Seq datasets are sparse and Patent requires a special split because all citations share one timestamp.
Experimental setup
Datasets: ML-20M Taobao Yelp GoogleLocal Flickr YouTube Patent WikiLink Wikipedia Reddit
Benchmarks: TGB-Seq (ML-20M, Taobao, Yelp, GoogleLocal, Flickr, YouTube, Patent, WikiLink) Wikipedia Reddit
Metrics: MRR AP training time per epoch
Baselines: JODIE DyRep TGAT TGN CAWN TCL GraphMixer DyGFormer EdgeBank SGNN-HN
Builds on: DyGLib TGB EdgeBank DyGFormer GraphMixer
Closest papers by meaning
- TGB 2.0: A Benchmark for Learning on Temporal Knowledge Graphs and Heterogeneous Graphs (NeurIPS 2024 · similarity 0.91)
- Improving Temporal Link Prediction via Temporal Walk Matrix Projection (NeurIPS 2024 · similarity 0.87)
- Temporal Graph Contrastive Learning for Sequential Recommendation (AAAI 2024 · similarity 0.87)
- State Space Models on Temporal Graphs: A First-Principles Study (NeurIPS 2024 · similarity 0.87)
- Using Time-Aware Graph Neural Networks to Predict Temporal Centralities in Dynamic Graphs (NeurIPS 2024 · similarity 0.87)
- DyG-Mamba: Continuous State Space Modeling on Dynamic Graphs (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.