Research map / Graph learning / GNN architectures
NLGT: Neighborhood-based and Label-enhanced Graph Transformer Framework for Node Classification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes NLGT, a graph transformer for node classification that combines hierarchical neighborhood sampling, label-enhanced feature fusion, neighborhood-based masked attention, and prediction voting over sampled subgraphs.
Problem
Message-passing GNNs suffer from oversmoothing and perform poorly on sparse graphs, while existing graph transformers add high complexity and use only shallow node features for node classification.
Approach
Serializes sampled subgraphs as unordered token sequences with a learnable [CLS] token, fuses label embeddings into node features, masks only [PAD] tokens in full self-attention over each subgraph, and votes over predictions from multiple sampled subgraphs at inference.
Key results
- NLGT reaches 84.03% test accuracy on Cora, 73.15% on Citeseer, 84.59% on Pubmed and 73.96% on Ogbn-Arxiv, the best among the compared methods in Table 2.
- Ablation on Ogbn-Arxiv: test accuracy rises from 71.63% (no components) to 73.96% with neighborhood mask attention, label-enhanced features and voting combined.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use NLGT instead of HEAL | Node classification · Cora | Test.Acc: 83.47 ± 0.17 -> 84.03 ± 0.15 | Helped |
| Use NLGT instead of HEAL | Node classification · Citeseer | Test.Acc: 72.76 ± 0.20 -> 73.15 ± 0.18 | Helped |
| Use NLGT instead of Polynormer | Node classification · Pubmed | Test.Acc: 83.63 ± 0.09 -> 84.59 ± 0.12 | Helped |
| Use NLGT instead of Polynormer | Node classification · Ogbn-Arxiv | Test.Acc: 73.36 ± 0.11 -> 73.96 ± 0.08 | Helped |
| Use neighborhood-based masked attention instead of adjacency-based masking or unmasked full attention (ablation) | NLGT · Cora, Citeseer, Pubmed, Arxiv | Accuracy: Cora: 83.12 ± 0.24, 83.41 ± 0.38 -> 84.03 ± 0.15; Citeseer: 71.13 ± 0.35, 72.45 ± 0.33 -> 73.15 ± 0.18; Pubmed: 83.41 ± | Helped |
| Add label-enhanced features (ablation) | NLGT without neighborhood-based attention or voting · Ogbn-Arxiv | Test Accuracy: 71.63 ± 0.26 -> 72.17 ± 0.23 | Helped |
| Add voting (ablation) | NLGT without neighborhood-based attention or label-enhanced features · Ogbn-Arxiv | Test Accuracy: 71.63 ± 0.26 -> 71.93 ± 0.21 | Helped |
| Add neighborhood-based attention (ablation) | NLGT without label-enhanced features or voting · Ogbn-Arxiv | Test Accuracy: 71.63 ± 0.26 -> 72.68 ± 0.18 | Helped |
| Add label-enhanced features (ablation) | NLGT with neighborhood-based attention and without voting · Ogbn-Arxiv | Test Accuracy: 72.68 ± 0.18 -> 73.28 ± 0.09 | Helped |
| Add voting (ablation) | NLGT with neighborhood-based attention and without label-enhanced features · Ogbn-Arxiv | Test Accuracy: 72.68 ± 0.18 -> 72.97 ± 0.16 | Helped |
| Add voting (ablation) | NLGT with neighborhood-based attention and label-enhanced features · Ogbn-Arxiv | Test Accuracy: 73.28 ± 0.09 -> 73.96 ± 0.08 | Helped |
Limitations
- Evaluated only on four node classification benchmarks, all homophilous citation/OGB graphs.
- Gains over strongest baselines are small (e.g., about 0.2-0.7 points), with no significance testing reported beyond std.
- Efficiency and complexity claims are not quantified.
Experimental setup
Datasets: Cora Citeseer Pubmed Ogbn-Arxiv
Benchmarks: Cora Citeseer Pubmed Ogbn-Arxiv
Metrics: test accuracy validation accuracy
Baselines: GCN GraphSAGE GAT GraphGPS Exphormer HEAL Polynormer
Builds on: Graphormer Exphormer Polynormer GAT BERT
Closest papers by meaning
- Classic GNNs are Strong Baselines: Reassessing GNNs for Node Classification (NeurIPS 2024 · similarity 0.89)
- Can Classic GNNs Be Strong Baselines for Graph-level Tasks? Simple Architectures Meet Excellence (ICML 2025 · similarity 0.89)
- Integrating Co-Training with Edge Discrimination to Enhance Graph Neural Networks Under Heterophily (AAAI 2025 · similarity 0.89)
- Label-free Node Classification on Graphs with Large Language Models (LLMs) (ICLR 2024 · similarity 0.89)
- Structural Information Enhanced Graph Representation for Link Prediction (AAAI 2024 · similarity 0.89)
- What Improves the Generalization of Graph Transformers? A Theoretical Dive into the Self-attention and Positional Encoding (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.