Research map / Graph learning / Graph self-supervised learning
Dynamic Bundling with Large Language Models for Zero-Shot Inference on Text-Attributed Graphs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DENSE, which samples topologically or semantically close node bundles, queries an LLM (GPT-4o) for each bundle's mode category, and supervises a GNN with entropy-based and ranking losses plus dynamic bundle refinement for zero-shot node classification on TAGs.
Problem
Zero-shot inference on text-attributed graphs with LLMs faces limited graph-structure information in prompts and unreliable (hallucinated or noisy) per-node LLM labels that corrupt GNN supervision.
Approach
Instead of querying LLMs with individual node texts, query them with bundles of proximate node texts for a mode-category label, supervise a GNN with bundle-level entropy and ranking losses, and iteratively evict low-confidence nodes from bundles.
Key results
- DENSE achieves the best accuracy on all ten datasets, e.g., 75.09 on Cora and 92.51 on Texas, versus second-best LLM-BP at 72.59 on Cora and 81.66 on Texas.
- Ablations show random bundle sampling causes severe drops on History (12 classes) and Sportsfit (13 classes), and individual supervision (L_IE) underperforms bundle supervision.
- Bundle queries are generally easier for the LLM than individual queries, with large gains on CiteSeer and Cornell.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use DENSE instead of the strongest reported baseline | Zero-shot TAG classification · Cora | accuracy: 72.59 -> 75.09 | Helped |
| use DENSE instead of the strongest reported baseline | Zero-shot TAG classification · CiteSeer | accuracy: 69.51 -> 72.37 | Helped |
| use DENSE instead of the strongest reported baseline | Zero-shot TAG classification · WikiCS | accuracy: 68.62 -> 71.03 | Helped |
| use DENSE instead of the strongest reported baseline | Zero-shot TAG classification · History | accuracy: 59.86 -> 67.31 | Helped |
| use DENSE instead of the strongest reported baseline | Zero-shot TAG classification · Children | accuracy: 30.76 -> 31.75 | Helped |
| use DENSE instead of the strongest reported baseline | Zero-shot TAG classification · Sportsfit | accuracy: 67.21 -> 75.88 | Helped |
| use DENSE instead of the strongest reported baseline | Zero-shot TAG classification · Cornell | accuracy: 83.28 -> 84.82 | Helped |
| use DENSE instead of the strongest reported baseline | Zero-shot TAG classification · Texas | accuracy: 81.66 -> 92.51 | Helped |
| use DENSE instead of the strongest reported baseline | Zero-shot TAG classification · Wisconsin | accuracy: 77.75 -> 87.17 | Helped |
| use DENSE instead of the strongest reported baseline | Zero-shot TAG classification · Washington | accuracy: 73.14 -> 81.66 | Helped |
| use topological or semantic proximity instead of random bundle sampling (ablation) | DENSE · 10 TAG datasets | accuracy | Helped |
| use bundle queries instead of individual node queries (ablation) | DENSE · 10 TAG datasets | accuracy | Helped |
Surprising findings
- Generative LLMs and graph foundation models perform inconsistently; several GNN-with-LLM methods (e.g., DGI, GraphMAE, OFA) score very low on many datasets.
Limitations
- Not applicable when node text is hard for LLMs to understand.
- Not directly applicable where GNNs are inherently weak on the graph structure.
Experimental setup
Datasets: Cora CiteSeer WikiCS History Children Sportsfit Cornell Texas Wisconsin Washington
Benchmarks: Cora CiteSeer WikiCS History Children Sportsfit Cornell Texas Wisconsin Washington
Metrics: accuracy
Baselines: SBERT RoBERTa TE-3-Large LLM2Vec GPT-3.5-turbo GPT-4o DGI GraphMAE OFA GOFA UniGLM ZeroG
Builds on: ranking loss LLM-BP
Closest papers by meaning
- Leveraging Large Language Models for Node Generation in Few-Shot Learning on Text-Attributed Graphs (AAAI 2025 · similarity 0.90)
- Generalization Principles for Inference over Text-Attributed Graphs with Large Language Models (ICML 2025 · similarity 0.89)
- Label-free Node Classification on Graphs with Large Language Models (LLMs) (ICLR 2024 · similarity 0.89)
- LLMs as Zero-shot Graph Learners: Alignment of GNN Representations with LLM Token Embeddings (NeurIPS 2024 · similarity 0.88)
- LoReC: Rethinking Large Language Models for Graph Data Analysis (NeurIPS 2026 · similarity 0.88)
- Harnessing Explanations: LLM-to-LM Interpreter for Enhanced Text-Attributed Graph Representation Learning (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.