Research map / Architectures and efficiency / CNN design
Continuous Rotation Group Equivariant Network Inspired by Neural Population Coding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CRGEN, a rotation group equivariant CNN whose kernels are generated from sparse learnable-location weights modulated by Gaussian tuning curves, inspired by neural population coding, achieving near-continuous rotation equivariance with far fewer parameters.
Problem
Discrete rotation group convolutions only achieve equivariance on a discrete set of angles, while inputs and transformations are continuous; prior continuous approaches rely on elaborate basis kernels or interpolation with smoothness assumptions.
Approach
Modulate group convolution kernels with Gaussian (bell-shaped) tuning curves, generate kernels from sparse weights with learnable continuous location and orientation parameters, and prove that discrete group convolutions with tuning-curve widths above one sampling step achieve continuous equivariance.
Key results
- CRGEN-S reaches 0.85% error on MNIST-rot without rotation augmentation with 40K parameters, versus 1.17% for E4-Net (41.4K) and 0.88% for SFCNN (6.5M).
- With rotation augmentation, CRGEN-L reaches 0.706% error with 150K parameters, beating PDO-eConv (0.709%, 650K).
- In small-sample MNIST-rot experiments, CRGEN-S improves over G-CNN by 1.61% to 24.12% in accuracy, reaching 80.17% with 200 samples.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CRGEN-S instead of SFCNN | MNIST-rot without rotation augmentation · MNIST-rot, without rotation augmentation | error (%): 0.88% vs. 0.85% | Helped |
| Use CRGEN-T instead of G-CNN | MNIST-rot without rotation augmentation · MNIST-rot, without rotation augmentation | error (%): 2.28% vs. 1.16% | Helped |
| Use CRGEN-L instead of PDO-eConv | MNIST-rot with rotation augmentation · MNIST-rot, with rotation augmentation | error (%): 0.709% vs. 0.706% | Helped |
| Use CRGEN-S instead of PDO-eConv | MNIST-rot with rotation augmentation · MNIST-rot, with rotation augmentation | error (%): 0.709% vs. 0.714% | Hurt |
| Use CRGEN-T instead of CNN | MNIST rotation generalization at p=0 · MNIST, randomly rotated test; training rotation probability p=0 | accuracy (%): 44.70% -> 93.38% | Helped |
| Use CRGEN instead of G-CNN | MNIST rotation generalization at p=1 · MNIST, randomly rotated test; training rotation probability p=1 | accuracy (%): 99.32% vs. 99.29% | No clear effect |
| Add CRGEN-LN to the plain architecture | MNIST · MNIST, randomly rotated test; training at 0° and 135° | accuracy (%): 96.23 -> 97.82 | Helped |
| Add CRGEN-LN to the ResNet-110 architecture | CIFAR10 · CIFAR10, randomly rotated test; training at 0° and 135° | accuracy (%): 61.98 -> 66.34 | Helped |
| Add CRGEN-LN to the ResNet-18 architecture | ImageNet-100 · ImageNet-100, randomly rotated test; training at 0° and 135° | accuracy (%): 50.83 -> 72.34 | Helped |
| Increase the number of orientations from 4 to 16 (ablation) | CRGEN-T · MNIST-rot | error (%): 2.72 -> 1.16 | Helped |
| Increase the number of orientations from 16 to 20 (ablation) | CRGEN-T · MNIST-rot | error (%): 1.16 -> 1.102 | Helped |
| Increase sparse points per kernel from 2 to 7 (ablation) | CRGEN-T · MNIST-rot | error (%): 1.26 -> 0.998 | Helped |
Surprising findings
- On ImageNet-100 with ResNet architecture, G-CNN (50.83%) underperforms plain CNN (57.80%), while CRGEN-LN reaches 72.34%.
Limitations
- Tuning curves reduce spatial/orientation resolution, especially in deep stacks of the proposed layers.
- Deeper networks suffer blurring; CRGEN-LN needs an equivariant loss and a G-CNN backbone to recover continuous equivariance.
- Sparse convolutions require additional engineering, so computational efficiency is not realized in the main method.
Experimental setup
Datasets: MNIST-rot MNIST CIFAR10 ImageNet-100
Benchmarks: MNIST-rot MNIST CIFAR10 ImageNet-100
Metrics: error rate (%) accuracy (%) parameter count
Baselines: CNN G-CNN PDO-eConv E4-Net SFCNN E2CNN ORN-8 TI-Pooling H-Net RotEqNet PTN-CNN
Builds on: G-CNN E2CNN PDO-eConv SFCNN E4-Net
Closest papers by meaning
- Group Equivariant Conditional Neural Processes (ICLR 2021 · similarity 0.88)
- Variational Partial Group Convolutions for Input-Aware Partial Equivariance of Rotations and Color-Shifts (ICML 2024 · similarity 0.88)
- Equivariant Neural Tangent Kernels (ICML 2025 · similarity 0.87)
- Group Downsampling with Equivariant Anti-aliasing (ICLR 2025 · similarity 0.87)
- Lie Group Decompositions for Equivariant Neural Networks (ICLR 2024 · similarity 0.87)
- Learning symmetries via weight-sharing with doubly stochastic tensors (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.