Research map / Architectures and efficiency / CNN design
Affine Equivariant Networks Based on Differential Invariants
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds affine-equivariant CNN layers (InvarPDEs-Net, InvarLayer) from differential invariants of symmetric PDEs, using SupNorm-normalized relative invariants to avoid division by zero, without discretizing or sampling the group.
Problem
Equivariant networks for the affine group usually require discretizing or sampling the group, causing model size and memory to grow with group size or depth; differential invariants of the affine group are fractional and can cause division-by-zero.
Approach
Treats feature extraction as evolution under symmetric PDEs built from differential invariants, and replaces fractional affine differential invariants with SupNorm-normalized polynomial relative differential invariants (SNDIs), giving equivariant layers whose size does not grow with group dimension.
Key results
- On affNIST (trained on MNIST, out-of-distribution), InvarLayer reaches 98.45% ± 0.15 accuracy vs 95.08% for affConv, a 3.37-point gain.
- On Scale-Fashion, InvarPDEs-Net achieves 89.62% ± 0.26 vs 87.48% for SE-CNN (best prior).
- On RS-MNIST, InvarPDEs-Net achieves 95.80% ± 0.09 vs 93.19% for RST-CNN.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use InvarPDEs-Net instead of SESN | Scale-MNIST classification · Scale-MNIST | accuracy: 97.92 ± 0.09 -> 98.30 ± 0.06 | Helped |
| Use InvarPDEs-Net instead of SE-CNN | Scale-Fashion classification · Scale-Fashion | accuracy: 87.48 -> 89.62 ± 0.26 | Helped |
| Use InvarLayer instead of SE-CNN | Scale-Fashion classification · Scale-Fashion | accuracy: 87.48 -> 89.50 ± 0.15 | Helped |
| Use InvarPDEs-Net instead of RST-CNN | Rotation-scale equivariant classification · RS-MNIST | accuracy: 93.19 ± 0.29 -> 95.80 ± 0.09 | Helped |
| Use InvarLayer instead of RST-CNN | Rotation-scale equivariant classification · RS-MNIST | accuracy: 93.19 ± 0.29 -> 93.15 ± 0.21 | No clear effect |
| Use InvarPDEs-Net instead of RST-CNN | Rotation-scale equivariant classification · RS-Fashion | accuracy: 78.64 ± 0.60 -> 79.48 ± 0.31 | Helped |
| Use InvarLayer instead of RST-CNN | Rotation-scale equivariant classification · RS-Fashion | accuracy: 78.64 ± 0.60 -> 74.51 ± 0.71 | Hurt |
| Use InvarPDEs-Net instead of affConv | Affine-equivariant classification · affNIST (out-of-distribution; trained on MNIST) | accuracy: 95.08 ± 0.31 -> 95.72 ± 0.12 | Helped |
| Use InvarLayer instead of RU CapsNet | Affine-equivariant classification · affNIST (out-of-distribution; trained on MNIST) | accuracy: 97.69 -> 98.45 ± 0.15 | Helped |
Surprising findings
- InvarLayer shows higher variability on RS-Fashion (74.51 ± 0.71), trailing RST-CNN, which the authors attribute to needing hyperparameter tuning.
Limitations
- Pooling introduces some equivariance error; the method is evaluated mainly on small-scale image classification.
- InvarLayer performance shows variability across setups (e.g., RS-Fashion).
- Theory covers Lie groups satisfying regularity conditions; only affine group and subgroups are implemented.
Experimental setup
Datasets: MNIST Fashion-MNIST Scale-MNIST Scale-Fashion RS-MNIST RS-Fashion affNIST
Benchmarks: Scale-MNIST Scale-Fashion RS-MNIST RS-Fashion affNIST
Metrics: test accuracy
Baselines: affConv CapsNet GE CapsNet affine CapsNet RU CapsNet SiCNN SI-ConvNet SEVF DSS SS-CNN SESN ScDCFNet
Builds on: learnable PDEs (Liu et al.) affConv steerable CNNs partial differential operators equivariance
Closest papers by meaning
- Affine Steerable Equivariant Layer for Canonicalization of Neural Networks (ICLR 2025 · similarity 0.92)
- Lie Group Decompositions for Equivariant Neural Networks (ICLR 2024 · similarity 0.90)
- On Making $SE(2)$-Invariant Networks Optimal (NeurIPS 2026 · similarity 0.89)
- A Characterization Theorem for Equivariant Networks with Point-wise Activations (ICLR 2024 · similarity 0.89)
- Equivariant Neural Tangent Kernels (ICML 2025 · similarity 0.88)
- Variational Partial Group Convolutions for Input-Aware Partial Equivariance of Rotations and Color-Shifts (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.