Research map / Graph learning / Knowledge and temporal graphs
Unifying Text Semantics and Graph Structures for Temporal Text-attributed Graphs with Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CROSS, a framework that extends existing TGNNs for temporal text-attributed graphs by using an LLM (DeepSeek-v2) to dynamically summarize evolving node neighborhoods and a semantic-structural co-encoder that mixes text and structure layer by layer.
Problem
Modeling temporal text-attributed graphs (TTAGs), where text semantics around nodes evolve over time and interact with graph structure; existing TGNNs embed text statically and fuse modalities only shallowly.
Approach
An LLM is prompted at sampled timestamps to summarize each node's textual neighborhood (temporal semantics extractor), and a co-encoder alternates Transformer-based semantic layers, TGNN structural layers and a cross-modal mixer so semantics and structure reinforce each other during encoding.
Key results
- CROSS with TGN backbone reaches 95.84% MRR on Enron transductive vs 73.05% for TGN, and average MRR gains of 22-26 points across TGAT, TGN, DyGFormer backbones.
- On the Industrial fraud node-classification task, CROSS gives an AUC gain of about 3.7% (per abstract).
- LLM zero-shot and one-shot DeepSeek-v2 link prediction is far below TGNN baselines (e.g., 24.18% and 46.27% MRR on Enron).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add CROSS to TGNN backbones | TGAT, TGN, and DyGFormer · Temporal link prediction, all five datasets and both settings | MRR (%): Avg.↑26.59; Avg.↑25.45; Avg.↑22.03 | Helped |
| Replace raw text with LLM-generated text in semantic encoding (ablation) | TGAT semantic encoding · Enron transductive | MRR (%): 49.73 ± 0.8 -> 63.07 ± 0.5; ↑13.34 | Helped |
| Replace raw text with LLM-generated text in semantic encoding (ablation) | TGAT semantic encoding · ICEWS1819 transductive | MRR (%): 77.45 ± 0.5 -> 85.04 ± 1.5; ↑7.59 | Helped |
| Replace raw text with LLM-generated text in structural encoding (ablation) | TGAT structural encoding · Enron transductive | MRR (%): 66.06 ± 0.1 -> 63.65 ± 1.7; ↓2.41 | Hurt |
| Replace raw text with LLM-generated text in structural encoding (ablation) | TGAT structural encoding · ICEWS1819 transductive | MRR (%): 85.81 ± 0.2 -> 86.12 ± 0.1; ↑0.31 | Helped |
| Use semantic-structural co-encoding with LLM-generated text rather than structural encoding with LLM-generated text (ablation) | TGAT backbone · Enron transductive | MRR (%): 63.65 ± 1.7 -> 95.58 ± 0.7 | Helped |
| Use semantic-structural co-encoding with LLM-generated text rather than structural encoding with LLM-generated text (ablation) | DyGFormer backbone · ICEWS1819 inductive | MRR (%): 77.70 ± 0.8 -> 87.80 ± 0.6 | Helped |
| Mix all semantic representations instead of only the latest semantic representation (ablation) | CROSS cross-modal mixer · Temporal link prediction | MRR (%) | No clear effect |
| Remove any single model component (ablation) | CROSS with TGN backbone · Temporal link prediction | MRR (%) | Hurt |
| Use DeepSeek-v2 rather than other tested LLM backbones (ablation) | CROSS on GDELT · GDELT | MRR (%) | Helped |
Surprising findings
- LLM-generated summaries give only marginal gains or slight degradation when used with structural encoding alone, suggesting structural encoding adds irrelevant high-order information.
- Mixing all historical semantic representations does not improve performance; using only the latest one works better.
Limitations
- Relies on paid LLM API calls (DeepSeek-v2) for summarization, limiting cost-efficiency at scale.
- Industrial dataset is private, limiting reproducibility.
- Cross-modal mixer uses only the latest semantic representation, discarding older temporal semantics.
Experimental setup
Datasets: Enron GDELT ICEWS1819 Googlemap_CT Industrial
Benchmarks: Enron GDELT ICEWS1819 Googlemap_CT Industrial
Metrics: MRR AUC AP
Baselines: JODIE DyRep TGAT TGN CAWN PINT TCL GraphMixer DyGFormer LKD4DyTAG FreeDyG DeepSeek-v2 zero/one-shot
Builds on: TGAT TGN DyGFormer MiniLM
Closest papers by meaning
- DTGB: A Comprehensive Benchmark for Dynamic Text-Attributed Graphs (NeurIPS 2024 · similarity 0.91)
- SSTAG: Structure-Aware Self-Supervised Learning Method for Text-Attributed Graphs (NeurIPS 2025 · similarity 0.89)
- TGB 2.0: A Benchmark for Learning on Temporal Knowledge Graphs and Heterogeneous Graphs (NeurIPS 2024 · similarity 0.89)
- TEG-DB: A Comprehensive Dataset and Benchmark of Textual-Edge Graphs (NeurIPS 2024 · similarity 0.88)
- Generalization Principles for Inference over Text-Attributed Graphs with Large Language Models (ICML 2025 · similarity 0.88)
- TGLsta: Low-resource Textual Graph Learning with Semantic and Topological Awareness via LLMs (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.