Research map / Graph learning / GNN architectures
Graph-less Neural Networks: Teaching Old MLPs New Tricks Via Distillation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Distill GraphSAGE predictions into an MLP | MLP node classifier · CPF datasets, transductive | accuracy: 21.32 (36.00%) on Cora; 12.16 (20.40%) on Citeseer; 7.87 (11.65%) on Pubmed; 15.23 (22.46%) on A-computer; 13.34 (16.94% | Helped |
| Distill GraphSAGE predictions into an MLP | MLP node classifier · Arxiv and Products, transductive | accuracy: 7.41 (13.24%) on Arxiv; 6.39 (10.23%) on Products | Helped |
| Distill GraphSAGE predictions into an MLP | MLP node classifier · Arxiv and Products, transductive | accuracy: -7.46 (-10.52%) on Arxiv; -9.75 (-12.4%) on Products | Hurt |
| Use a wider distilled MLP (ablation) | GLNN · Arxiv and Products, transductive | accuracy: Arxiv: 63.46 $\pm$ 0.45 -> 72.15 $\pm$ 0.27; Products: 68.86 $\pm$ 0.46 -> 77.65 $\pm$ 0.48 | Helped |
| Replace GraphSAGE teacher with MLP+C&S (ablation) | GLNN+ · Products | accuracy: 77.65 -> 82.94 | Helped |
| Add one-hop neighbor feature augmentation (ablation) | GLNN · Arxiv inductive | accuracy: 60.48 -> 68.83 | Helped |
| Add one-hop neighbor feature augmentation (ablation) | MLP · Arxiv inductive | accuracy: 55.40 -> 66.62 | Helped |
| Increase Gaussian noise in node features (ablation) | GLNN · CPF datasets, inductive | accuracy | Hurt |
| Increase the inductive share of the evaluation split (ablation) | GLNN · Production setting across CPF datasets | accuracy | Hurt |
| Use GCN, GAT, or APPNP instead of GraphSAGE as teacher (ablation) | GLNN · CPF datasets | accuracy | No clear effect |
| Use APPNP instead of GraphSAGE as teacher (ablation) | GLNN · CPF datasets | accuracy | Hurt |
| Set the hard-label loss weight to a nonzero value (ablation) | GLNN distillation objective · Node classification experiments | accuracy | No clear effect |
Closest papers by meaning
- TINED: GNNs-to-MLPs by Teacher Injection and Dirichlet Energy Distillation (ICML 2025 · similarity 0.92)
- VQGraph: Rethinking Graph Representation Space for Bridging GNNs and MLPs (ICLR 2024 · similarity 0.91)
- Classic GNNs are Strong Baselines: Reassessing GNNs for Node Classification (NeurIPS 2024 · similarity 0.91)
- Diffusing to the Top: Boost Graph Neural Networks with Minimal Hyperparameter Tuning (ICLR 2025 · similarity 0.91)
- Can Classic GNNs Be Strong Baselines for Graph-level Tasks? Simple Architectures Meet Excellence (ICML 2025 · similarity 0.90)
- Forward Learning of Graph Neural Networks (ICLR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.