Research map / Graph learning / GNN architectures

LLM Enhancers for GNNs: An Analysis from the Perspective of Causal Mechanism Identification

ICML 2025 · Graph learning › GNN architectures · approach: GNN generalization theory · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Analyzes the LLM-enhancer-plus-GNN paradigm via interchange interventions on a synthetic causal graph dataset (CCSG), derives three empirical findings, and proposes an attention-based transmission (AT) module that selects LLM output token features, improving GCN/GAT/GraphSAGE on Cora, Pubmed, and Instagram.

Problem

The fundamental properties and internal mechanisms of the LLM-enhancer-plus-GNN paradigm, where LLM-generated node features feed a GNN, remain underexplored.

Approach

Applies interchange intervention (causal abstraction) with a controllable causal synthetic graph dataset (CCSG) to align GNN hidden variables with a high-order causal model, and uses the findings to motivate an attention-based selection of LLM output token features passed to the GNN.

Key results

• The AT module improves accuracy across all tested LLM/GNN combinations, e.g., +2.03 on Cora with Qwen2+GCN and +3.09 on Pubmed with Llama3+GraphSAGE.
• Node- and graph-level interchange interventions show the LLM enhancer's features align with node-level and raw-data-level variables, with GNN alignments appearing in shallow layers.
• Optimal alignment loss L_II is lower for models with higher accuracy, suggesting L_II partially reflects model capability.

Measured effects reported in the paper

Surprising findings

• Scaling up the GNN only enlarges its structure without improving causal relation modeling, according to the authors' hypothesis.
• Changing which LLM token positions are passed to the GNN affects performance more than model scale or LLM choice in their analysis.

Limitations

• Analysis focuses on fixed-parameter LLM enhancers and mainly on GNN internals; LLM causal modeling not analyzed.
• Assumes node as smallest variable unit, ignoring internal node interactions.
• Theoretical results rely on assumption that both models map graphs to outputs exactly, which holds only approximately in practice.

Experimental setup

Datasets: CCSG Cora Pubmed Instagram

Benchmarks: Cora Pubmed Instagram

Metrics: accuracy L_II interchange intervention loss

Baselines: Llama2 Qwen2 Llama3

Builds on: interchange intervention causal abstraction TAPE

Closest papers by meaning

• $\texttt{G1}$: Teaching LLMs to Reason on Graphs with Reinforcement Learning (NeurIPS 2025 · similarity 0.89)
• Harnessing Explanations: LLM-to-LM Interpreter for Enhanced Text-Attributed Graph Representation Learning (ICLR 2024 · similarity 0.89)
• Rethinking Causal Relationships Learning in Graph Neural Networks (AAAI 2024 · similarity 0.89)
• Attention Mechanisms Perspective: Exploring LLM Processing of Graph-Structured Data (ICML 2025 · similarity 0.88)
• Label-free Node Classification on Graphs with Large Language Models (LLMs) (ICLR 2024 · similarity 0.88)
• Graph Neural Prompting with Large Language Models (AAAI 2024 · similarity 0.88)