Research map / Graph learning / Graph self-supervised learning
TEG-DB: A Comprehensive Dataset and Benchmark of Textual-Edge Graphs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces TEG-DB, a benchmark of nine textual-edge graph datasets (text on both nodes and edges) and benchmarks PLM-, GNN-, entangled-GNN- and LLM-as-predictor methods on link prediction and node classification.
Problem
Existing text-attributed graph datasets put text only on nodes, with edges reduced to binary or categorical attributes, so methods cannot exploit rich textual edge information and experimental settings are inconsistent.
Approach
Provides a unified textual-edge graph format with nine datasets from book, e-commerce, academic and social domains, plus an entangled PLM encoding that embeds node and edge texts jointly before message passing.
Key results
- GraphSAGE/GeneralConv/GINE/GraphTransformer and other GNNs outperform the worst PLM/GNN method by roughly 5-20% on the datasets, with the best method varying by dataset.
- Using edge text improves link prediction AUC by at least about 3% and F1 by at least about 8% across all datasets relative to not using edge text.
- LLM-as-predictor methods (GPT-3.5-TURBO, GPT-4) perform poorly on link prediction, e.g. GPT-4 reaches AUC 0.8780 on Goodreads-Children versus about 0.95+ for GNNs on the same dataset.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Entangle edge text and node text before GPT encoding instead of encoding text directly (ablation) | TEG link prediction methods · All evaluated datasets | AUC: about 2% | Helped |
| Entangle edge text and node text before GPT encoding instead of encoding text directly (ablation) | TEG node classification methods · All evaluated datasets | Micro-AUC: about 2% | Helped |
| Include edge text instead of using no edge text (ablation) | TEG link prediction methods · All evaluated datasets | AUC: at least approximately a 3% | Helped |
| Include edge text instead of using no edge text (ablation) | TEG link prediction methods · All evaluated datasets | F1: at least approximately an 8% | Helped |
| Include edge text instead of using no edge text (ablation) | TEG node classification methods · All evaluated datasets | Micro-AUC: at least a 3% | Helped |
| Include edge text instead of using no edge text (ablation) | TEG node classification methods · All evaluated datasets | Micro-F1: a 6% | Helped |
| Use BERT text embeddings instead of no text embeddings (ablation) | TEG link prediction methods · All evaluated datasets | AUC: approximately 5% | Helped |
| Use BERT-Large or GPT-3.5-TURBO text embeddings instead of no text embeddings (ablation) | TEG link prediction methods · All evaluated datasets | AUC: about 7% | Helped |
| Use BERT text embeddings instead of no text embeddings (ablation) | TEG node classification methods · All evaluated datasets | Micro-AUC: approximately 3% | Helped |
| Use BERT-Large or GPT-3.5-TURBO text embeddings instead of no text embeddings (ablation) | TEG node classification methods · All evaluated datasets | Micro-AUC: about 3.5% | Helped |
| Use LLMs as direct predictors rather than PLM-based or GNN-based methods | TEG link prediction · All evaluated datasets | AUC: approximately 10% - 30% | Hurt |
Surprising findings
- LLM-as-predictor methods such as GPT-4 perform near chance on several link prediction datasets (e.g., AUC about 0.50 on Amazon-Movie).
- EdgeGNN performs markedly worse than other GNNs on the Citation node classification task (ACC about 0.41-0.43 versus around 0.80 for others).
Limitations
- LLM-as-predictor evaluation limited to GPT-3.5-TURBO and GPT-4 via API, with weak performance attributed to difficulty of understanding graph topology.
- Entangled encoding evaluated only with GPT-3.5-TURBO embeddings.
- Hardware and compute details are deferred to an appendix not included in the provided text.
Experimental setup
Datasets: Goodreads Amazon Reddit Twitter MAG Semantic Scholar Open Data Platform
Benchmarks: TEG-DB Goodreads-Children TEG-DB Goodreads-Crime TEG-DB Amazon-Apps TEG-DB Amazon-Movie TEG-DB Citation TEG-DB Twitter
Metrics: AUC F1 ACC Micro-AUC Micro-F1
Baselines: MLP GraphSAGE GeneralConv GINE EdgeGNN GraphTransformer GPT-3.5-TURBO GPT-4
Builds on: GraphSAGE GINE BERT GraphTransformer
Closest papers by meaning
- DTGB: A Comprehensive Benchmark for Dynamic Text-Attributed Graphs (NeurIPS 2024 · similarity 0.91)
- TGB 2.0: A Benchmark for Learning on Temporal Knowledge Graphs and Heterogeneous Graphs (NeurIPS 2024 · similarity 0.89)
- Can Large Language Models Analyze Graphs like Professionals? A Benchmark, Datasets and Models (NeurIPS 2024 · similarity 0.88)
- Unifying Text Semantics and Graph Structures for Temporal Text-attributed Graphs with Large Language Models (NeurIPS 2025 · similarity 0.88)
- TGLsta: Low-resource Textual Graph Learning with Semantic and Topological Awareness via LLMs (AAAI 2025 · similarity 0.88)
- Talk like a Graph: Encoding Graphs for Large Language Models (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.