Research map / Graph learning / GNN architectures
VQGraph: Rethinking Graph Representation Space for Bridging GNNs and MLPs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes VQGraph, a structure-aware graph VQ-VAE tokenizer whose codebook labels nodes' local substructures, and distills soft code assignments from a GNN teacher into an MLP student for GNN-to-MLP distillation.
Problem
GNN-to-MLP distillation transfers only class-level predictions, which are not expressive enough to capture diverse local graph structures, limiting the accuracy of fast, graph-free MLP students.
Approach
Learn a graph VQ-VAE tokenizer (GraphSAGE encoder plus codebook, with attribute, topology and label reconstruction) so each node's local substructure becomes a discrete code, then distill the teacher's soft code assignments (KL over codebook distances) to the MLP in addition to soft labels.
Key results
- VQGraph averages 3.90% accuracy gain over its GraphSAGE teacher and 1.39% over NOSMOG across seven datasets in the standard transductive setting.
- On Citeseer, VQGraph reaches 76.08% accuracy versus 73.78% for NOSMOG and 58.50% for MLP, with inference about 828x faster than 3-layer GraphSAGE.
- Average cut value of VQGraph is 0.9493, higher than NOSMOG (0.9348) and GraphSAGE (0.9276).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace class-label representation space with structure-aware codebook and soft-code-assignment distillation | GNN-to-MLP distillation · Seven standard graph datasets | accuracy: ↑ 1.39% | Helped |
| replace class-label representation space with structure-aware codebook and soft-code-assignment distillation | GNN-to-MLP distillation · Seven standard graph datasets | accuracy: 3.90% | Helped |
| use structure-aware VQ tokenizer while retaining soft-label distillation (ablation) | class-based GNN-to-MLP distillation · Seven standard graph datasets | accuracy: ↑ 5.25%; ↑ 3.00%; ↑ 2.77%; ↑ 1.63%; ↑ 2.08%; ↑ 10.98%; ↑ 16.25% | Helped |
| add classic graph autoencoder structure awareness (ablation) | class-based GNN-to-MLP distillation · Seven standard graph datasets | accuracy | Helped |
| replace classic graph autoencoder with graph VQ-VAE tokenizer (ablation) | class-based GNN-to-MLP distillation · Seven standard graph datasets | accuracy | Helped |
| add soft code-assignment distillation to the VQ tokenizer (ablation) | Only-VQ · Seven standard graph datasets | accuracy: ↑ 6.82%; ↑ 3.71%; ↑ 4.57%; ↑ 2.97%; ↑ 2.45%; ↑ 13.61%; ↑ 16.93% | Helped |
| use L2 distance instead of cosine similarity for code relations (ablation) | VQGraph · Seven standard graph datasets | accuracy: approximately 0.2% on average accuracy | Helped |
| vary codebook size (ablation) | VQGraph · Graph datasets | accuracy | Mixed |
| increase hidden size of NOSMOG (ablation) | NOSMOG · Citeseer | accuracy | Mixed |
Surprising findings
- Only-VQ (teacher trained with VQ but distilled with soft labels only) already beats class-based GLNN on most datasets, so the tokenizer alone helps.
Limitations
- Codebook size must be tuned per dataset (e.g., 2048 to 32768), with larger graphs needing larger codebooks.
- Large-scale tokenizer training relies on GraphSAGE subgraph sampling, which only approximately captures global structure.
- Heterophilic experiments are limited to two small datasets (Texas and Cornell).
Experimental setup
Datasets: Citeseer Pubmed Cora A-computer A-photo Arxiv Products Texas Cornell
Benchmarks: Citeseer Pubmed Cora A-computer A-photo Arxiv Products Texas Cornell
Metrics: accuracy inference time (ms) cut value
Baselines: GraphSAGE teacher MLP GLNN NOSMOG Graph-MLP GCN GAT APPNP AE+Class-based
Builds on: VQ-VAE GLNN NOSMOG CPF
Closest papers by meaning
- Demystifying GNN-to-MLP Knowledge Transfer: Theoretical Grounding and Dual-Stream Distillation Method (AAAI 2026 · similarity 0.92)
- Extracting Low-/High- Frequency Knowledge from Graph Neural Networks and Injecting It into MLPs: An Effective GNN-to-MLP Distillation Framework (AAAI 2023 · similarity 0.91)
- Graph-less Neural Networks: Teaching Old MLPs New Tricks Via Distillation (ICLR 2022 · similarity 0.91)
- Distilling Graph Geometry: Knowledge Gap from GNNs to MLPs (NeurIPS 2026 · similarity 0.90)
- Large Language Model Meets Graph Neural Network in Knowledge Distillation (AAAI 2025 · similarity 0.89)
- TINED: GNNs-to-MLPs by Teacher Injection and Dirichlet Energy Distillation (ICML 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.