Research map / Graph learning / GNN architectures

Fast Graph Sharpness-Aware Minimization for Enhancing and Accelerating Few-Shot Node Classification

NeurIPS 2024 · Graph learning › GNN architectures · approach: GNN generalization theory

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes FGSAM and FGSAM+, which perturb GNN parameters with message passing but minimize the perturbed loss with MP-free MLPs (PeerMLPs), reusing the perturbation gradient to inject topology, to get SAM-level generalization for few-shot node classification at lower cost than SAM and sometimes Adam.

Problem

Few-shot node classification requires GNNs that generalize to unseen classes from few labels, but SAM improves generalization at twice the training cost of base optimizers, which is especially costly for GNNs.

Approach

Use the GNN (with message passing) only for the SAM perturbation step and an MLP (same weights, no message passing) to minimize the perturbed loss, reuse the perturbation gradient to reintroduce topology cheaply, and periodically compute exact updates (FGSAM+) while approximating intermediate steps with reused gradients.

Key results

• On FSNC, FGSAM+ with Meta-GCN reaches 67.71% average accuracy vs 64.32% for Adam and 64.75% for SAM, with training time 6.51s vs 9.48s (Adam) and 19.03s (SAM) on CoraFull.
• FGSAM+ with TENT trains in 15.10s vs 22.03s for SAM on CoraFull average, and FGSAM+ with GPN averages 63.30% vs 58.15% for Adam on CoraFull.
• On standard node classification, FGSAM with GraphSAGE averages 70.05% vs 68.40% for Adam across nine datasets.

Measured effects reported in the paper

Surprising findings

• PeerMLP-style MP-free minimization outperforms standard SAM and Adam in most FSNC settings despite dropping message passing during minimization.
• ESAM trains slower than SAM on graphs because the per-iteration sample-loss sorting is non-negligible for small GNNs.

Limitations

• For models with many non-GNN components (e.g., TENT), FGSAM+ training time can remain longer than Adam.
• Theoretical justification of MP-free minimization is limited to a linear case on synthetic CSBM graphs.
• Hyperparameter search (Optuna) and experiments limited to small GNN depths; some ablations only on GPN with CoraFull.

Experimental setup

Datasets: CoraFull DBLP ogbn-arXiv Cora Citeseer Pubmed Chameleon Squirrel Actor Cornell Texas Wisconsin

Benchmarks: CoraFull DBLP ogbn-arXiv Cora Citeseer Pubmed Chameleon Squirrel Actor Cornell Texas Wisconsin

Metrics: accuracy training time (sec. per 200 episodes)

Baselines: Adam SAM ESAM LookSAM AE-SAM Meta-GCN AMM-GNN GPN TENT GCN GAT GraphSAGE

Builds on: Sharpness-Aware Minimization LookSAM MAML GPN PeerMLP

Closest papers by meaning

• Classic GNNs are Strong Baselines: Reassessing GNNs for Node Classification (NeurIPS 2024 · similarity 0.89)
• Beyond Real-world Benchmark Datasets: An Empirical Study of Node Classification with GNNs (NeurIPS 2022 · similarity 0.89)
• Beyond Fixed Depth: Adaptive Graph Neural Networks for Node Classification Under Varying Homophily (AAAI 2026 · similarity 0.89)
• Normalize Then Propagate: Efficient Homophilous Regularization for Few-Shot Semi-Supervised Node Classification (AAAI 2025 · similarity 0.88)
• Diffusing to the Top: Boost Graph Neural Networks with Minimal Hyperparameter Tuning (ICLR 2025 · similarity 0.88)
• Bonsai: Gradient-free Graph Condensation for Node Classification (ICLR 2025 · similarity 0.87)