Research map / Architectures and efficiency / CNN design
Equivariant Neural Tangent Kernels
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Derives neural tangent kernel and NNGP recursions for group convolutional networks, proves infinite-width equivalence between data augmentation of non-equivariant nets and GCNNs, and shows equivariant NTKs beat non-equivariant kernels on histology and QM9.
Problem
Training dynamics of equivariant neural networks are poorly understood, in particular how they compare to data-augmented training of non-equivariant networks.
Approach
Layer-wise recursive relations for NNGP and NTK of group convolutional layers, lifting layers, nonlinearities and group pooling; these show that infinite-width networks trained with full group augmentation share the same mean predictions as GCNNs, at all training times and off-manifold.
Key results
- Equivariant (C4 roto-translation) NTK kernel predictors outperform non-equivariant CNN NTK predictors on histological image classification across training set sizes.
- SO(3)-invariant GCNN NTK gives a considerable MAE improvement over MLP NTK for QM9 U0 energy prediction across training set sizes.
- Finite-width ensembles of CNNs trained with rotation augmentation approach the outputs of C4 GCNN ensembles on MNIST, including out-of-distribution inputs, as ensemble size grows.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a C4 roto-translation-equivariant GCNN NTK instead of a CNN NTK | Kernel prediction for histological image classification · Histological image classification | test accuracy | Helped |
| Use an SO(3)-invariant GCNN NTK instead of an MLP NTK | Kernel prediction of molecular energies · QM9 | MAE | Helped |
| Train a CNN with rotation augmentation rather than an unaugmented rotation-equivariant GCNN | Infinite-width ensemble mean predictions · Fully group-augmented training data | prediction | No clear effect |
| Increase the ensemble size for CNNs trained with augmentation and equivariant GCNNs | Finite-width ensemble predictions · MNIST, including out-of-distribution data | L2 distance between logits | Helped |
Limitations
- Theoretical equivalence holds only in the infinite-width limit; finite-width ensembles match only approximately.
- SO(3) Fourier implementation restricted to global filter supports due to lack of efficient 3j-symbol computation.
Experimental setup
Datasets: histological image dataset (NCT-CRC-HE style, nine tissue classes) QM9 MNIST CIFAR10
Benchmarks: histological image classification QM9 U0 energy regression MNIST out-of-distribution ensemble convergence CIFAR10 ensemble convergence
Metrics: test accuracy mean absolute error L2 distance between logits
Baselines: non-equivariant CNN NTK MLP NTK
Builds on: neural tangent kernel neural network gaussian process group convolutional neural networks neural-tangents
Closest papers by meaning
- Lie Group Decompositions for Equivariant Neural Networks (ICLR 2024 · similarity 0.89)
- A Tale of Two Symmetries: Exploring the Loss Landscape of Equivariant Models (NeurIPS 2025 · similarity 0.89)
- Approximately Equivariant Neural Processes (NeurIPS 2024 · similarity 0.88)
- Affine Equivariant Networks Based on Differential Invariants (CVPR 2024 · similarity 0.88)
- Neural Fourier Transform: A General Approach to Equivariant Representation Learning (ICLR 2024 · similarity 0.88)
- Probing Equivariance and Symmetry Breaking in Convolutional Networks (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.