Research map / Graph learning / Graph self-supervised learning

Beyond the Golden Teacher: Enhancing Graph Learning through LLM-GNN Co-teaching

NeurIPS 2026 · Graph learning › Graph self-supervised learning · approach: LLMs and graph foundation models · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes LLM-GNN Co-Teaching, a bidirectional co-teaching framework where a GNN and an LLM iteratively exchange small-loss pseudo-labels and the LLM is further tuned with DPO on cross-round self-correction pairs (RPL-PO), for few-shot node classification on text-attributed graphs.

Problem

Few-shot semi-supervised node classification on text-attributed graphs, where GNNs fail on cold nodes and LLMs fail on text-ambiguous nodes, and existing LLM-GNN methods use one model as a fixed golden teacher.

Approach

Replace the fixed golden teacher with heterogeneous co-teaching: GNN and LLM exchange confident pseudo-labels under architecture-specific small-loss selection and both update each round; LLM preference pairs are mined from nodes that move from cross-model disagreement to agreement across rounds, requiring no human labels or reward model.

Key results

• 3-shot: 85.75% on Cora vs 77.89% for GNN-as-Judge (+7.86 points).
• 3-shot: 69.94% on ogbn-arxiv vs 62.21% for GNN-as-Judge (+7.73 points).
• Ablation: removing bidirectional teaching drops Cora 3-shot accuracy from 85.75% to 78.66%; removing RPL-PO drops it to 83.03%.

Surprising findings

• Agreement-based selection is outperformed by small-loss ranking on most datasets, indicating confidence ranking adds value beyond simple agreement filtering.

Limitations

• Runtime grows linearly with number of co-teaching rounds, roughly 2.6x GNN-as-Judge runtime on ogbn-arxiv.
• Evaluated only on text-rich citation, Wikipedia and Amazon graphs; not validated where node text is noisy, sparse, or absent.
• Confident LLM mistakes can persist across rounds, and the loop may amplify biases.

Experimental setup

Datasets: Cora Citeseer PubMed WikiCS ogbn-arxiv ogbn-products (subset)

Benchmarks: Cora Citeseer PubMed WikiCS ogbn-arxiv ogbn-products (subset)

Metrics: node classification accuracy

Baselines: GCN GAT GraphSAGE Zero-Shot Graph-CoT GLEM TAPE LLM-GNN LLaGA GraphGPT GNN-as-Judge

Builds on: Co-Teaching GNN-as-Judge DPO TAPE GLEM

Closest papers by meaning

• Preference-driven Knowledge Distillation for Few-shot Node Classification (NeurIPS 2025 · similarity 0.89)
• Label-free Node Classification on Graphs with Large Language Models (LLMs) (ICLR 2024 · similarity 0.89)
• SSTAG: Structure-Aware Self-Supervised Learning Method for Text-Attributed Graphs (NeurIPS 2025 · similarity 0.88)
• Large Language Model Meets Graph Neural Network in Knowledge Distillation (AAAI 2025 · similarity 0.88)
• Dynamic Bundling with Large Language Models for Zero-Shot Inference on Text-Attributed Graphs (NeurIPS 2025 · similarity 0.88)
• Harnessing Explanations: LLM-to-LM Interpreter for Enhanced Text-Attributed Graph Representation Learning (ICLR 2024 · similarity 0.88)