Research map / Graph learning / GNN architectures
Fast Track to Winning Tickets: Repowering One-Shot Pruning for Graph Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FastGLT, a one-shot pruning plus gradual denoising framework that finds graph lottery tickets with higher sparsity and far lower search cost than iterative magnitude pruning methods.
Problem
Identifying graph lottery tickets (joint sparse subgraphs and subnetworks of GNNs) with iterative magnitude pruning is very computationally expensive, while one-shot pruning typically underperforms IMP.
Approach
Treat one-shot pruned tickets as a fast track to winning tickets: after one-shot pruning to near-target sparsity, gradually denoise the masks by swapping out noisy low-magnitude weights/edges for potential ones identified by accumulated gradients (weights) and edge degree (edges) during training.
Key results
- FastGLT finds graph lottery tickets with up to 45.62% higher weight sparsity and 22.71% higher graph sparsity than WD-GLT on small graphs.
- On Ogbn-Arxiv with ResGCN, FastGLT reaches 70.25% weight sparsity or 48.01% graph sparsity, exceeding UGS by 30.18% and 36.82%.
- FastGLT is 1.7-44x faster than IMP-based GLT methods and achieves 95.3%-98.6% MAC savings.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FastGLT instead of WD-GLT | GCN · Cora | accuracy: 80.27 ± 0.45 -> 80.33 ± 0.17 | Helped |
| Use FastGLT instead of WD-GLT | GCN · Citeseer | accuracy: 70.74 ± 0.51 -> 70.59 ± 0.12 | Hurt |
| Use FastGLT instead of WD-GLT | GCN · PubMed | accuracy: 78.82 ± 0.33 -> 79.11 ± 0.29 | Helped |
| Use FastGLT instead of WD-GLT | GIN · Cora | accuracy: 78.26 ± 0.44 -> 78.32 ± 0.17 | Helped |
| Use FastGLT instead of WD-GLT | GIN · Citeseer | accuracy: 68.50 ± 0.38 -> 68.54 ± 0.25 | Helped |
| Use FastGLT instead of WD-GLT | GIN · PubMed | accuracy: 77.80 ± 0.35 -> 77.59 ± 0.37 | Hurt |
| Use FastGLT instead of WD-GLT | GAT · Cora | accuracy: 80.11 ± 0.61 -> 80.07 ± 0.37 | Hurt |
| Use FastGLT instead of WD-GLT | GAT · Citeseer | accuracy: 69.38 ± 1.23 -> 69.30 ± 0.21 | Hurt |
| Use FastGLT instead of WD-GLT | GAT · PubMed | accuracy: 78.52 ± 0.20 -> 78.56 ± 0.68 | Helped |
| Start denoising from a one-shot ticket rather than a randomly initialized ticket (ablation) | FastGLT · Citeseer | weight sparsity: 94.22 -> 99.27; 89.38 -> 98.33; 70.26 -> 99.20 | Helped |
| Start denoising from a one-shot ticket rather than a randomly initialized ticket (ablation) | FastGLT · Citeseer | graph sparsity: 23.16 -> 34.17; 20.89 -> 45.06; 56.06 -> 68.97 | Helped |
| Vary denoising interval ΔT among 3, 5, 10, 20, 30, and 50 (ablation) | FastGLT · Citeseer and Ogbn-Arxiv | accuracy: 1.35%; 1.74% | No clear effect |
Surprising findings
- Denoising from randomly initialized tickets instead of one-shot tickets drops weight sparsity by up to 28.94% and graph sparsity by up to 24.17%.
Limitations
- Relies on one-shot pruning being close to IMP quality, which the paper validates only empirically on a limited set of datasets and backbones.
- Gradual denoising is sensitive to hyperparameters such as the intermediate sparsity function and denoising schedule, though the paper reports low sensitivity.
Experimental setup
Datasets: Cora Citeseer PubMed Ogbn-Arxiv Ogbn-Proteins Ogbl-Collab
Benchmarks: Cora Citeseer PubMed Ogbn-Arxiv Ogbn-Proteins Ogbl-Collab
Metrics: accuracy ROC-AUC Hits@50 weight sparsity graph sparsity wall-clock time inference MACs
Baselines: UGS WD-GLT random pruning one-shot pruning
Builds on: Lottery Ticket Hypothesis Graph Lottery Ticket (UGS) iterative magnitude pruning WD-GLT
Closest papers by meaning
- Searching Lottery Tickets in Graph Neural Networks: A Dual Perspective (ICLR 2023 · similarity 0.94)
- Rethinking Graph Lottery Tickets: Graph Sparsity Matters (ICLR 2023 · similarity 0.94)
- A Unified Lottery Ticket Hypothesis for Graph Neural Networks (ICML 2021 · similarity 0.93)
- TEDDY: Trimming Edges with Degree-based Discrimination Strategy (ICLR 2024 · similarity 0.92)
- GDeR: Safeguarding Efficiency, Balancing, and Robustness via Prototypical Graph Pruning (NeurIPS 2024 · similarity 0.89)
- Weisfeiler and Leman Go Gambling: Why Expressive Lottery Tickets Win (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.