Research map / Graph learning / Graph self-supervised learning
Actions Speak Louder than Prompts: A Large-Scale Study of LLMs for Graph Inference
Ben Finkelshtein, Silviu Cucerzan, Sujay Kumar Jauhar, Ryen W White
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Graph-as-Code instead of prompting on long-text homophilic graphs | o4-mini node classification · Photo / Wiki-CS / Citeseer | accuracy: Photo 85.60 (budget prompt) -> 86.40; Wiki-CS 80.80 -> 82.20 | Improved |
| increase neighborhood from 0-hop to 2-hop prompting (ablation) | Prompting, short-text homophilic · Cora | accuracy: 64.17 -> 83.43 | Improved |
| 2-hop prompting on dense or long-text graphs | Prompting · Products, Texas, Photo, Wiki-CS | accuracy: TokenLimit (fails) | Hurt |
| give LLM more agency: GraphTool to GraphTool+ to Graph-as-Code (ablation) | ReAct-style methods · Cora | accuracy: 74.02 -> 81.40 -> 85.16 | Improved |
| use LLM modes on heterophilic graphs (vs label propagation / majority) | all LLM modes · Cornell, Washington, Wisconsin | accuracy: Cornell Label propagation 41.74 -> Graph-as-Code 92.70 | Improved |
| Graph-as-Code vs label propagation on Texas | Texas · Texas (heterophilic) | accuracy: 78.90 -> 73.60 | Hurt |
| add 2-hop summary prompting (iterative neighborhood summarization) (ablation) | Prompting on long-text datasets · Wiki-CS | accuracy: 80.80 (budget) -> 81.70 | Improved |
| extend summarization from 2 to 4 hops (ablation) | Summary prompting · Wiki-CS / Reddit | accuracy: Wiki-CS 81.70 -> 80.90; Reddit 57.60 -> 55.90 | Hurt |
| scale LLM from Llama to GPT-5 (ablation) | Graph-as-Code · 7-dataset average | accuracy: 67.80 -> 82.71 | Improved |
| enable reasoning mode (ablation) | Graph-as-Code (Phi-4) · 7-dataset average | accuracy: 75.38 -> 83.19 | Improved |
| randomly shuffle adjacency matrix (destroy structure) (ablation) | 2-hop prompt vs Graph-as-Code · Cora | accuracy: 2-hop prompt 83.43 -> 44.50; Graph-as-Code 85.16 -> 56.50 | Hurt |
| use Graph-as-Code instead of prompting for shortest path | synthetic shortest path task · Erdos-Renyi graphs (50 nodes) | MSE: 2.82 -> 0.00 | Improved |
Closest papers by meaning
- Can Large Language Models Analyze Graphs like Professionals? A Benchmark, Datasets and Models (NeurIPS 2024 · similarity 0.91)
- Attention Mechanisms Perspective: Exploring LLM Processing of Graph-Structured Data (ICML 2025 · similarity 0.90)
- Talk like a Graph: Encoding Graphs for Large Language Models (ICLR 2024 · similarity 0.90)
- LoReC: Rethinking Large Language Models for Graph Data Analysis (NeurIPS 2026 · similarity 0.90)
- GraphRouter: A Graph-based Router for LLM Selections (ICLR 2025 · similarity 0.89)
- How Do Large Language Models Understand Graph Patterns? A Benchmark for Graph Pattern Comprehension (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
