Research map / Graph learning / GNN architectures
On Transferring Transferability: Towards a Theory for Size Generalization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds a consistent-sequence framework showing that size transferability of dimension-independent models is equivalent to continuity on a limit space, yielding size-generalization bounds and transferable DeepSet/PointNet/IGN/GNN variants.
Problem
Under what properties of the model, data, and learning task does performance trained on small-dimensional inputs transfer to larger inputs (size/dimension generalization) for dimension-independent architectures such as GNNs, DeepSets, and PointNet?
Approach
Formalizes size changes as consistent sequences of vector spaces with embeddings (zero-padding or duplication) and compatible norms, defines a limit space, and shows transferability is equivalent to (locally) Lipschitz continuity of compatible maps in that limit space; this yields a covering-number generalization bound and design principles for transferable architectures.
Key results
- Normalized DeepSet has best in-distribution and size-generalization on the population-statistics (mutual information) task, aligned at continuity level.
- Continuous GGNN achieves best test performance on large graphs for signal-weighted triangle density, while message-passing GNN performs poorly due to limited expressiveness.
- Normalized DS-CI and normalized SVD-DeepSet achieve good size generalization on ModelNet10 Gromov-Wasserstein lower bound; SVD-DS underperforms DS-CI but is more time and memory efficient.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use normalized DeepSet instead of PointNet or DeepSet | Population Statistics size-generalization task · Population Statistics | test MSE | Helped |
| Use PointNet instead of normalized DeepSet or DeepSet | Maximal Distance from the Origin size-generalization task · Maximal Distance from the Origin | test MSE | Helped |
| Use continuous GGNN instead of message-passing GNN | Graph size-generalization task · Attributed graphs | test MSE | Helped |
| Use normalized DS-CI or normalized SVD-DS | Point-cloud size-generalization task · ModelNet10 chair-sofa pairs | test MSE | Helped |
| Replace normalized DS-CI with normalized SVD-DS | Point-cloud size-generalization task · ModelNet10 chair-sofa pairs | test MSE | Hurt |
| Replace normalized DS-CI with normalized SVD-DS | Point-cloud model · ModelNet10 chair-sofa pairs | time and memory efficiency | Helped |
Surprising findings
- DeepSet with zero-padding is incompatible under the duplication sequence, so sum aggregation fails to transfer across set sizes in the probability-measure limit.
Limitations
- Some models are only locally Lipschitz outside a measure-zero set (e.g., SVD-based canonicalization), so transferability holds only almost everywhere.
- Experiments are small-scale on synthetic data and a single ModelNet10 task.
Experimental setup
Datasets: ModelNet10 synthetic Gaussian-mixture sets synthetic random attributed graphs
Benchmarks: ModelNet10
Metrics: mean test MSE
Baselines: DeepSet normalized DeepSet PointNet message-passing GNN DS-CI IGN
Builds on: graphon transferability for GNNs representation stability DeepSets PointNet invariant graph networks
Closest papers by meaning
- Transfer Learning of Graph Neural Networks with Ego-graph Information Maximization (NeurIPS 2021 · similarity 0.87)
- Features are fate: a theory of transfer learning in high-dimensional regression (ICML 2025 · similarity 0.87)
- How Much Can Transfer? BRIDGE: Bounded Multi-Domain Graph Foundation Model with Generalization Guarantees (ICML 2025 · similarity 0.87)
- Graph neural networks and non-commuting operators (NeurIPS 2024 · similarity 0.86)
- Contextual Structure Knowledge Transfer for Graph Neural Networks (AAAI 2025 · similarity 0.86)
- GraphBridge: Towards Arbitrary Transfer Learning in GNNs (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.