Research map / Graph learning / GNN architectures
Label Attentive Distillation for GNN-Based Graph Classification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LAD-GNN, a label-attentive teacher–student distillation training scheme that injects ground-truth label information into GNN node embeddings for graph classification, improving several GNN backbones on 10 TU datasets.
Problem
Conventional GNNs aggregate node-level features without graph-label information, causing misaligned node embeddings that hurt graph-level classification accuracy.
Approach
A teacher GNN with a label-attentive encoder (label embedding fused with node embeddings via dot-product attention) produces an ideal embedding that serves as intermediate MSE supervision for a label-free student GNN sharing the classification head; at inference only the student is used.
Key results
- LAD-GNN improves each of 7 GNN backbones, with about 3.3% average absolute accuracy gain on the 10 datasets.
- With GraphSAGE as backbone, LAD-GNN achieves 16.6% (IMDB-BINARY) and 16.8% (NCI1) accuracy improvements.
- Compared with 9 augmentation and distillation methods, LAD-GNN achieves the best results on all datasets except MUTAG, where IGSD is better.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply LAD-GNN to GraphSAGE | GraphSAGE · MUTAG | accuracy: 0.816±0.09 -> 0.863±0.10 | Helped |
| Apply LAD-GNN to GraphSAGE | GraphSAGE · COLLAB | accuracy: 0.791±0.05 -> 0.793±0.05 | No clear effect |
| Apply LAD-GNN to GraphSAGE | GraphSAGE · ENZYMES | accuracy: 0.742±0.09 -> 0.792±0.10 | Helped |
| Apply LAD-GNN to GraphSAGE | GraphSAGE · IMDB-BINARY | accuracy: 0.589±0.06 -> 0.755±0.08 | Helped |
| Apply LAD-GNN to GraphSAGE | GraphSAGE · IMDB-MULTI | accuracy: 0.420±0.05 -> 0.420±0.06 | No clear effect |
| Apply LAD-GNN to GraphSAGE | GraphSAGE · NCI1 | accuracy: 0.699±0.04 -> 0.867±0.07 | Helped |
| Apply LAD-GNN to GraphSAGE | GraphSAGE · PROTEINS | accuracy: 0.706±0.04 -> 0.733±0.05 | Helped |
| Apply LAD-GNN to GraphSAGE | GraphSAGE · PTC | accuracy: 0.709±0.07 -> 0.814±0.10 | Helped |
| Apply LAD-GNN to GraphSAGE | GraphSAGE · REDDIT-BINARY | accuracy: 0.831±0.05 -> 0.925±0.02 | Helped |
| Apply LAD-GNN to GraphSAGE | GraphSAGE · REDDIT-MULTI-5K | accuracy: 0.466±0.02 -> 0.517±0.04 | Helped |
| Use LAD-GNN instead of IGSD | Graph classification methods · MUTAG | accuracy: 0.902±0.070 -> 0.899±0.113 | Hurt |
Surprising findings
- A large fraction of node embeddings are negatively correlated with ground-truth label embeddings across standard GNN backbones (Fig. 1c).
Limitations
- Distillation loss adds training-time cost relative to the plain backbone.
- Gains on MUTAG are not best; IGSD outperforms LAD-GNN there.
- Hyperparameter lambda is sensitive and must be tuned per dataset on the validation set.
Experimental setup
Datasets: MUTAG PTC NCI1 PROTEINS ENZYMES COLLAB IMDB-BINARY IMDB-MULTI REDDIT-BINARY REDDIT-MULTI-5K
Benchmarks: MUTAG PTC NCI1 PROTEINS ENZYMES COLLAB IMDB-BINARY IMDB-MULTI REDDIT-BINARY REDDIT-MULTI-5K NCI109
Metrics: accuracy AUC
Baselines: GCN GAT GraphSAGE GIN DGCNN SAGPool MEWISPool DropEdge M-Mixup G-Mixup JOAOv2 AD-GCL
Builds on: knowledge distillation Transformer scaled dot-product attention GCN GIN MEWISPool
Closest papers by meaning
- Combining Label Propagation and Simple Models out-performs Graph Neural Networks (ICLR 2021 · similarity 0.89)
- NLGT: Neighborhood-based and Label-enhanced Graph Transformer Framework for Node Classification (AAAI 2025 · similarity 0.88)
- Preference-driven Knowledge Distillation for Few-shot Node Classification (NeurIPS 2025 · similarity 0.87)
- VQGraph: Rethinking Graph Representation Space for Bridging GNNs and MLPs (ICLR 2024 · similarity 0.87)
- Label-free Node Classification on Graphs with Large Language Models (LLMs) (ICLR 2024 · similarity 0.86)
- Noisy Node Classification by Bi-level Optimization Based Multi-Teacher Distillation (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.