Research map / Architectures and efficiency / CNN design
Learning symmetries via weight-sharing with doubly stochastic tensors
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Learns per-layer weight-sharing over kernels via learnable doubly stochastic (Sinkhorn-normalized) soft-permutation tensors, jointly optimized with the downstream task, so symmetries are discovered rather than fixed in advance.
Problem
Group-equivariant CNNs require the symmetry group to be specified beforehand, and fixed equivariance can over-constrain models when symmetries are partial or unknown.
Approach
Parameterize the regular-representation stack as learnable doubly stochastic matrices (via Sinkhorn iterations) acting as soft permutations on learnable base kernels, so weight sharing is learned jointly with the task and reduces to regular group convolution when symmetries are exact.
Key results
- On rotated MNIST, WSCNN + norm + ent reaches 98.04% vs 98.96% for C4-GCNN and 98.48% for CNN; on scaled MNIST WSCNN reaches 99.27% vs 97.50% for C4-GCNN.
- On CIFAR-10, WSCNN + norm reaches 78.80% with 1.63M params (+468K), comparable to CNN-128 (78.83% at 6.5M params).
- On CIFAR-10 with horizontal flips, WSCNN reaches 82.38% vs 76.05% for C4-GCNN, close to CNN-128 at 82.60%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use learned weight-sharing instead of fixed C4 equivariance | WSCNN vs GCNN · MNIST scaling | accuracy: 97.50 ± .15 -> 99.27 ± .04 | Helped |
| Use learned weight-sharing instead of a standard CNN | WSCNN + norm vs CNN · MNIST rotations | accuracy: 98.48 ± .08 -> 97.56 ± .07 | Hurt |
| Use learned weight-sharing instead of a standard CNN | WSCNN + norm vs CNN · MNIST scaling | accuracy: 99.30 ± .01 -> 99.27 ± .04 | No clear effect |
| Use learned weight-sharing instead of fixed C4 equivariance | WSCNN + norm vs GCNN · MNIST rotations | accuracy: 98.96 ± .14 -> 97.56 ± .07 | Hurt |
| Add entropy regularization (ablation) | WSCNN + norm · MNIST rotations | accuracy: 97.56 ± .07 -> 98.04 ± .11 | Helped |
| Add entropy regularization (ablation) | WSCNN + norm · MNIST scaling | accuracy: 99.27 ± .04 -> 99.24± .01 | No clear effect |
| Add entropy regularization (ablation) | WSCNN + norm · CIFAR-10 | accuracy: 78.80 ± 0.46 -> 76.80 ± 1.40 | Hurt |
| Use learned weight-sharing instead of fixed C4 equivariance | WSCNN vs GCNN · MNIST partial rotations [0, 90°] | accuracy: 98.84 ± .002 -> 98.87 ± .001 | No clear effect |
| Use learned weight-sharing instead of fixed C4 equivariance | WSCNN vs GCNN · MNIST partial rotations [0, 180°] | accuracy: 98.72± .001 -> 99.25 ± .001 | Helped |
| Use learned weight-sharing instead of fixed C4 equivariance | WSCNN vs GCNN · CIFAR-10 | accuracy: 76.72 ± 0.26 -> 78.80 ± 0.46 | Helped |
| Use learned weight-sharing instead of an unconstrained CNN-128 | WSCNN + norm vs CNN-128 · CIFAR-10 | accuracy: 78.83 ± 0.01 -> 78.80 ± 0.46 | No clear effect |
| Use learned weight-sharing instead of fixed C4 equivariance | WSCNN vs GCNN · CIFAR-10 horizontal flips | accuracy: 76.05 ± .004 -> 82.38 ± .003 | Helped |
Surprising findings
- The entropy regularizer did not help on scaled MNIST and in some settings reduced accuracy (e.g., CIFAR-10 76.80% vs 78.80% with norm only).
- The learned representations need not form an interpretable group structure; the method does not enforce associativity or closure.
Limitations
- Computational cost scales quadratically with group size and kernel support size.
- Representation stack is uniform across channels, so transformations such as color jitter cannot be learned.
- Entropy regularization needs task-specific tuning, and later-layer representations show low diversity.
Experimental setup
Datasets: MNIST (rotated) MNIST (scaled) CIFAR-10 CIFAR-10 (horizontal flips) toy 1D shift, 2D C8 rotation, 3D cube symmetry datasets
Benchmarks: RotatedMNIST scaled MNIST CIFAR-10 CIFAR-10 horizontal flips MNIST partial rotations
Metrics: test accuracy
Baselines: CNN C4-GCNN CNN-32 CNN-64 CNN-128
Builds on: group equivariant CNN Sinkhorn operator Birkhoff's theorem Efficient weight-sharing (Sheng et al. style learned weight-sharing matrices)
Closest papers by meaning
- A Tale of Two Symmetries: Exploring the Loss Landscape of Equivariant Models (NeurIPS 2025 · similarity 0.90)
- Improving Equivariant Networks with Probabilistic Symmetry Breaking (ICLR 2025 · similarity 0.90)
- Probing Equivariance and Symmetry Breaking in Convolutional Networks (NeurIPS 2025 · similarity 0.90)
- Lie Group Decompositions for Equivariant Neural Networks (ICLR 2024 · similarity 0.89)
- Permutation Equivariant Neural Networks for Symmetric Tensors (ICML 2025 · similarity 0.89)
- MatrixNet: Learning over symmetry groups using learned group representations (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.