Research map / Graph learning / GNN architectures
Unlocking the Potential of Black-box Pre-trained GNNs for Graph Few-shot Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Meta-BP, a lightweight graph meta-learner over frozen black-box pre-trained GNN (DGI) outputs, trained with an information-bottleneck objective and lottery-ticket-style subnetwork pruning for graph few-shot node classification.
Problem
Few-shot node classification when the pre-trained GNN is a black box (only node representations are returned; no parameters or gradients accessible), with task gap between pre-training and downstream tasks and limited meta-training task diversity.
Approach
A lightweight graph meta-learner takes frozen black-box node embeddings plus neighbor-averaged embeddings, is meta-trained with an information-bottleneck objective (neural mutual-information estimator) to keep only task-relevant knowledge, and is pruned via learnable masks (lottery ticket style) to a subnetwork used for fast adaptation at meta-test time.
Key results
- Meta-BP achieves the best accuracy in all reported settings, e.g., 66.38% on Cora 2-way 1-shot and 75.29% on Cora 2-way 3-shot (vs. best baseline TEG at 63.14% and TLP at 73.38%).
- On Cora-full 5-way 1-shot, Meta-BP reaches 66.05% vs. TEG 63.73%; on OGBN-arxiv 10-way 1-shot it reaches 43.78% vs. TEG 41.89%.
- Ablations show removing neighbor abstraction, information-bottleneck extraction, or subnetwork pruning each lowers accuracy; performance vs. capacity ratio peaks at intermediate values.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add neighbor abstractions to use neighboring node representations (ablation) | Meta-BP · Cora and OGBN-arxiv few-shot node classification | accuracy | Helped |
| Add minimal sufficient information extraction (ablation) | Meta-BP · Cora and OGBN-arxiv few-shot node classification | accuracy | Helped |
| Prune the graph meta-learner and transfer its subnetwork instead of the full parameter set (ablation) | Meta-BP · Cora and OGBN-arxiv few-shot node classification | accuracy | Helped |
| Vary the subnetwork capacity ratio; excessively small or large capacity can reduce performance (ablation) | Meta-BP · Cora and OGBN-arxiv few-shot node classification | accuracy | Mixed |
| Use GMI instead of DGI as the black-box pre-trained GNN (ablation) | Meta-BP · Cora-full few-shot node classification | accuracy: 66.05(±1.46) -> 66.13(±1.48); 72.98(±1.86) -> 75.58(±1.23); 51.41(±1.91) -> 51.88±(1.44); 57.79(±2.16) -> 62.28(±1.86) | Mixed |
| Use BGRL instead of DGI as the black-box pre-trained GNN (ablation) | Meta-BP · Cora-full few-shot node classification | accuracy: 66.05(±1.46) -> 65.92(±1.62); 72.98(±1.86) -> 73.64(±1.42); 51.41(±1.91) -> 51.36±(1.65); 57.79(±2.16) -> 59.47(±1.94) | Mixed |
Surprising findings
- Only the TLP baseline and Meta-BP operate in the black-box setting; many white-box baselines that update GNN parameters are still reported as comparisons.
Limitations
- Experiments only on node classification with a single DGI backbone in main results; other pre-trained GNNs tested only in one table on Cora-full.
- Black-box baselines are scarce (only TLP), so comparison is mostly against white-box methods that have more access.
- Performance depends on pre-training strategy quality and on hyperparameters such as capacity ratio and beta.
Experimental setup
Datasets: Cora Amazon Computers Cora-full OGBN-arxiv
Benchmarks: Cora Amazon Computers Cora-full OGBN-arxiv
Metrics: few-shot node classification accuracy (%)
Baselines: GCN GraphSage GMI DGI PN MAML Meta-SGC GPN G-Meta TLP TENT TEG
Builds on: MAML DGI Lottery Ticket Hypothesis Donsker-Varadhan mutual information estimator (MINE) information bottleneck
Closest papers by meaning
- LLMs as Zero-shot Graph Learners: Alignment of GNN Representations with LLM Token Embeddings (NeurIPS 2024 · similarity 0.88)
- Fast Graph Sharpness-Aware Minimization for Enhancing and Accelerating Few-Shot Node Classification (NeurIPS 2024 · similarity 0.87)
- HGPrompt: Bridging Homogeneous and Heterogeneous Graphs for Few-Shot Prompt Learning (AAAI 2024 · similarity 0.87)
- Preference-driven Knowledge Distillation for Few-shot Node Classification (NeurIPS 2025 · similarity 0.87)
- Graph Few-Shot Learning via Adaptive Spectrum Experts and Cross-Set Distribution Calibration (NeurIPS 2025 · similarity 0.87)
- Edge Prompt Tuning for Graph Neural Networks (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.