Research map / Graph learning / Knowledge and temporal graphs
DTGB: A Comprehensive Benchmark for Dynamic Text-Attributed Graphs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces DTGB, the first benchmark of eight large-scale dynamic text-attributed graphs with four tasks, evaluating seven dynamic graph models (with BERT text embeddings) and six LLMs on textual relation generation.
Problem
Existing dynamic graph datasets lack raw text attributes and edge categories, and TAG datasets lack temporal information, so there is no benchmark for dynamic text-attributed graphs (DyTAGs).
Approach
Construct eight time-evolving DyTAGs with node and edge text and edge categories, and define four tasks (future link prediction, destination node retrieval, edge classification, textual relation generation) that jointly require structural and textual modeling.
Key results
- Bert-initialized text improves AUC-ROC for most models in inductive future link prediction, with larger gains in the inductive setting.
- Fine-tuning Vicuna-7b with LoRA on 10k samples raises BERTScore F1 on Googlemap CT from 81.85 to 85.67.
- Existing dynamic graph models perform poorly on edge classification for GDELT and ICEWS1819 (weighted F1 around 0.10-0.32).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add BERT-encoded text initialization to dynamic graph models (ablation) | Dynamic graph models · Edge classification across DTGB datasets | Weighted F1 | Helped |
| Add BERT-encoded text initialization (ablation) | Dynamic graph models · Future link prediction, transductive | AUC-ROC | Mixed |
| Add BERT-encoded text initialization (ablation) | Dynamic graph models · Future link prediction, inductive | AUC-ROC | Helped |
| Add BERT-encoded text initialization (ablation) | Dynamic graph models · Destination node retrieval, transductive and inductive | Hits@10 | Helped |
| Use historical rather than random candidate sampling (ablation) | Dynamic graph models · Destination node retrieval | Hits@10 | Hurt |
| Add supervised fine-tuning on 10,000 interaction pairs (ablation) | Llama3-8b · Googlemap CT | BERTScore F1: 81.12 -> 81.84; 0.72$\uparrow$ | Helped |
| Add supervised fine-tuning on 10,000 interaction pairs (ablation) | Llama3-8b · Stack elec | BERTScore F1: 81.09 -> 81.97; 0.88$\uparrow$ | Helped |
| Add supervised fine-tuning on 10,000 interaction pairs (ablation) | Vicuna-7b · Googlemap CT | BERTScore F1: 81.85 -> 85.67; 3.82$\uparrow$ | Helped |
| Add supervised fine-tuning on 10,000 interaction pairs (ablation) | Vicuna-7b · Stack elec | BERTScore F1: 81.46 -> 82.67; 1.21$\uparrow$ | Helped |
| Add supervised fine-tuning on 10,000 interaction pairs (ablation) | Vicuna-13b · Googlemap CT | BERTScore F1: 81.99 -> 84.67; 2.68$\uparrow$ | Helped |
| Add supervised fine-tuning on 10,000 interaction pairs (ablation) | Vicuna-13b · Stack elec | BERTScore F1: 81.33 -> 82.73; 1.40$\uparrow$ | Helped |
| Add BERT-encoded text initialization (ablation) | TCL · Amazon movies edge classification | Weighted F1 | Hurt |
Surprising findings
- Adding Bert-encoded text embeddings degrades memory-based models (JODIE, DyRep) on future link prediction in several datasets.
- Open-source LLMs such as Mistral-7B and Vicuna perform comparably to GPT-3.5-turbo and GPT-4o on textual relation generation.
Limitations
- Textual relation generation does not incorporate high-order graph context due to LLM input length limits.
- Text is encoded with frozen BERT initialization, a preliminary strategy for text integration into dynamic graph models.
- Memory-based models (JODIE, DyRep) face scalability issues on large graphs and produce OOM on several datasets.
Experimental setup
Datasets: Enron GDELT ICEWS1819 Stack elec Stack ubuntu Googlemap CT Amazon movies Yelp
Benchmarks: DTGB
Metrics: Weighted Precision Weighted Recall Weighted F1 AP AUC-ROC Hits@k BERTScore
Baselines: JODIE DyRep TGAT CAWN TCL GraphMixer DyGFormer Mistral-7B Vicuna-7B Vicuna-13B Llama-3-8B GPT-3.5-turbo
Builds on: DyGLib DyGFormer TGB
Closest papers by meaning
- TEG-DB: A Comprehensive Dataset and Benchmark of Textual-Edge Graphs (NeurIPS 2024 · similarity 0.91)
- Unifying Text Semantics and Graph Structures for Temporal Text-attributed Graphs with Large Language Models (NeurIPS 2025 · similarity 0.91)
- TGB 2.0: A Benchmark for Learning on Temporal Knowledge Graphs and Heterogeneous Graphs (NeurIPS 2024 · similarity 0.90)
- SSTAG: Structure-Aware Self-Supervised Learning Method for Text-Attributed Graphs (NeurIPS 2025 · similarity 0.88)
- Generalization Principles for Inference over Text-Attributed Graphs with Large Language Models (ICML 2025 · similarity 0.88)
- TGLsta: Low-resource Textual Graph Learning with Semantic and Topological Awareness via LLMs (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.