Research map / Graph learning / Graph self-supervised learning

LoReC: Rethinking Large Language Models for Graph Data Analysis

NeurIPS 2026 · Graph learning › Graph self-supervised learning · approach: LLMs and graph foundation models · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes LoReC, a training-free, plug-and-play decoding framework (Look: attention redistribution; Remember: graph re-injection into FFN; Contrast: logit rectification) that improves GraphLLM predictions on node classification over GraphGPT and GraphPrompter.

Problem

GraphLLMs that feed graph tokens into LLMs for node prediction show limited or even degraded performance versus GNNs, because LLMs under-attend graph tokens and over-rely on textual priors.

Approach

A three-stage training-free decoding method: uncertainty-triggered attention amplification toward graph tokens, uncertainty-triggered injection of graph tokens as key-value memory into FFN layers, and contrastive logit subtraction against text-only and perturbed-graph logits.

Key results

• LoReC improves GraphGPT accuracy by up to 2.58% and GraphPrompter by up to 5.12%, with macro-F1 gains up to 4.24% and 7.97%.
• LoReC-GraphPrompter (Llama2) reaches 0.8137 accuracy on Cora versus 0.7915 for GraphPrompter (Llama2-7b).
• Ablation on Arxiv shows each of Look, Remember, and Contrast modules gives independent gains, and the full combination is best.

Surprising findings

• Text-only Qwen3-8b achieves performance comparable to GraphLLM models on some datasets, suggesting predictions are driven mainly by textual priors.
• Scaling text features degrades performance more than scaling graph features, indicating over-reliance on text.

Limitations

• Latency and memory overhead is higher than base GraphPrompter (82.69 vs 45.23 ms/token).
• Hyperparameters (entropy threshold, eta, alpha, omega, beta, edge threshold) are tuned per dataset and base model.
• Gains on some settings are marginal, e.g. Citeseer accuracy for the Llama2 variant is lower than GraphPrompter's text-only-scored baseline in macro-F1.

Experimental setup

Datasets: PubMed Cora Citeseer ogbn-arxiv ogbn-products

Benchmarks: Cora Citeseer PubMed ogbn-arxiv ogbn-products

Metrics: Accuracy Macro-F1 Micro-F1

Baselines: GCN GAT GKD GLNN Llama2-7b Vicuna-7b Baichuan-7b Qwen3-8b GraphGPT GraphPrompter

Builds on: GraphGPT GraphPrompter contrastive decoding adaptive degree-based graph augmentation

Closest papers by meaning

• GLBench: A Comprehensive Benchmark for Graph with Large Language Models (NeurIPS 2024 · similarity 0.92)
• Can Large Language Models Analyze Graphs like Professionals? A Benchmark, Datasets and Models (NeurIPS 2024 · similarity 0.91)
• How Do Large Language Models Understand Graph Patterns? A Benchmark for Graph Pattern Comprehension (ICLR 2025 · similarity 0.91)
• LLaGA: Large Language and Graph Assistant (ICML 2024 · similarity 0.91)
• Talk like a Graph: Encoding Graphs for Large Language Models (ICLR 2024 · similarity 0.90)
• Attention Mechanisms Perspective: Exploring LLM Processing of Graph-Structured Data (ICML 2025 · similarity 0.90)