Research map / Graph learning / GNN architectures
Scale Equivariant Graph Metanetworks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ScaleGMN, a graph metanetwork that is equivariant to both permutation and scaling symmetries of feedforward/CNN parameters, applied to INR classification/editing and generalization prediction.
Problem
Weight-space networks (metanetworks) that process neural network parameters only respect permutation symmetries, ignoring scaling symmetries induced by activation functions such as ReLU, tanh and sine.
Approach
Builds scale-equivariant message passing from new building blocks (canonicalisation/symmetrisation for scale invariance, rescale-equivariant Hadamard products for products of multipliers) so vertex and edge representations transform equivariantly under permutations and valid scalings; proves it can simulate FFNN forward and backward passes.
Key results
- ScaleGMN-B reaches 96.59% on MNIST INR classification and 80.78% on F-MNIST, versus 91.40% and 68.94% for the prior best baselines.
- On generalization prediction (CIFAR-10-GS ReLU), ScaleGMN-B achieves Kendall tau 0.9282 versus 0.9270 for NFN-HNP.
- On MNIST INR dilation, ScaleGMN-B reaches MSE 1.89e-2, beating all permutation-equivariant baselines (best 2.06 for NG-GNN-64).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use scale-equivariant ScaleGMN rather than the strongest listed baseline | INR classification metanetwork · MNIST INR | accuracy: 91.40 ± 0.60 -> 96.59 ± 0.24 | Helped |
| Use scale-equivariant ScaleGMN rather than the strongest listed baseline | INR classification metanetwork · F-MNIST INR | accuracy: 68.94 ± 0.64 -> 80.78 ± 0.16 | Helped |
| Use scale-equivariant ScaleGMN rather than the strongest listed baseline | INR classification metanetwork · CIFAR-10 INR | accuracy: 36.04 ± 0.44 -> 38.82 ± 0.10 | Helped |
| Use scale-equivariant ScaleGMN rather than the strongest listed baseline | INR classification metanetwork · Augmented CIFAR-10 INR | accuracy: 46.60 ± 0.07 -> 56.95 ± 0.57 | Helped |
| Use scale-equivariant ScaleGMN rather than the strongest listed baseline | Generalization prediction metanetwork · CIFAR-10-GS ReLU | Kendall-τ: 0.9270 ± 0.001 -> 0.9282 ± 0.003 | Helped |
| Use scale-equivariant ScaleGMN rather than the strongest listed baseline | Generalization prediction metanetwork · SVHN-GS ReLU | Kendall-τ: 0.8636 ± 0.002 -> 0.8689 ± 0.003 | Helped |
| Use scale-equivariant ScaleGMN rather than the strongest listed baseline | Generalization prediction metanetwork · CIFAR-10-GS Tanh | Kendall-τ: 0.9340 ± 0.001 -> 0.9425 ± 0.004 | Helped |
| Use scale-equivariant ScaleGMN rather than the strongest listed baseline | Generalization prediction metanetwork · SVHN-GS Tanh | Kendall-τ: 0.8620 ± 0.003 -> 0.8736 ± 0.003 | Helped |
| Use scale-equivariant ScaleGMN rather than the strongest listed baseline | Generalization prediction metanetwork · CIFAR-10-GS both activations | Kendall-τ: 0.934 ± 0.001 -> 0.941 ± 0.000 | Helped |
| Add bidirectional message passing (ablation) | ScaleGMN · Dilating MNIST INR | MSE in 10^-2: 2.56 ± 0.03 -> 1.89 ± 0.00 | Helped |
| Add scaling data augmentation (ablation) | Permutation-equivariant baselines · F-MNIST INR | accuracy: 67.06 ±0.29 -> 71.42±0.45; 68.19 ± 0.28 -> 70.34±0.13; 68.94 ± 0.64 -> 70.24 ±0.47; 68.0±0.20 -> 72.01 ± 1.4 | Helped |
| Add positive-scaling data augmentation (ablation) | Permutation-equivariant baselines · CIFAR-10-GS ReLU | Kendall-τ: 0.9190 ± 0.010 -> 0.8474 ± 0.01; 0.9270 ± 0.001 -> 0.8906±0.01; 0.9010±0.060 -> 0.8855±0.02 | Hurt |
Surprising findings
- Simply adding scaling augmentations to permutation-equivariant baselines did not surpass the original baselines for positive-scale (ReLU) symmetry, showing positive scaling augmentation is hard to design.
Limitations
- Designed only for feedforward and convolutional networks; does not cover skip connections, softmax or normalisation layers and their symmetries.
- Complete characterisation of functions expressible by scale-equivariant building blocks and of overall expressive power remains open.
- Bidirectional variant relies on edge inversion or reciprocal features, which caused numerical instabilities; positive-scale augmentation not effective.
Experimental setup
Datasets: MNIST INR F-MNIST INR CIFAR-10 INR Augmented CIFAR-10 INR Small CNN Zoo CIFAR-10-GS Small CNN Zoo SVHN-GS MNIST INR dilation
Benchmarks: MNIST INR classification F-MNIST INR classification CIFAR-10 INR classification Augmented CIFAR-10 INR classification CIFAR-10-GS generalization prediction SVHN-GS generalization prediction MNIST INR dilation editing
Metrics: test accuracy Kendall's tau MSE
Baselines: MLP Inr2Vec DWS NFN-NP NFN-HNP NG-GNN StatNN
Builds on: Deep Weight Space networks (DWS) Neural Graphziggurat / NG-GNN Neural Functional Networks (NFN) SignNet Sign equivariant networks
Closest papers by meaning
- A Tale of Two Symmetries: Exploring the Loss Landscape of Equivariant Models (NeurIPS 2025 · similarity 0.89)
- Improving Equivariant Networks with Probabilistic Symmetry Breaking (ICLR 2025 · similarity 0.89)
- On the hardness of learning under symmetries (ICLR 2024 · similarity 0.89)
- A Characterization Theorem for Equivariant Networks with Point-wise Activations (ICLR 2024 · similarity 0.89)
- Graph Metanetworks for Processing Diverse Neural Architectures (ICLR 2024 · similarity 0.88)
- Are High-Degree Representations Really Unnecessary in Equivariant Graph Neural Networks? (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.