Research map / Learning theory and algorithms / Sample complexity
Synergy and Symmetry in Deep Learning: Interactions between the Data, Model, and Inference Algorithm
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply O(3)^d (spurious) rotation to data, breaking the weight-sharing symmetry (ablation) | GAP / VEC models on CIFAR-10 · CIFAR-10 | test accuracy | Hurt |
| Switch inference from NTK regression to finite-width SGD with larger LR and L2 (NN+) (ablation) | GAP without spurious symmetry · CIFAR-10 | test accuracy: 77% to 84% | Helped |
| Switch inference from NTK to finite-width SGD with larger LR and L2 (NN+) (ablation) | VEC without spurious symmetry · CIFAR-10 | test accuracy: 67% to 78% | Helped |
| Use SGD instead of NTK inference when data or model has spurious symmetry (O(3)^d) (ablation) | LCN/VEC with O(3)^d-rotated data · CIFAR-10 | test accuracy | No clear effect |
| Increase finite width of VEC beyond ~2^6-2^7 (NN/NN+) without rotation (ablation) | VEC_n · CIFAR-10 | test accuracy | Hurt |
| Add L2 regularization and larger learning rate (NN+) to VEC_n (ablation) | VEC_n at small width · CIFAR-10 | validation accuracy variance | Mixed |
| Rotate ImageNet data by O(3)^d, increasing training-set size (DIDE scaling) (ablation) | ResNet101 on ImageNet · ImageNet | power-law slope alpha in second phase: alpha 0.49 vs 0.41 for clean baseline | No clear effect |
| Scale ResNet from ResNet-18 to ResNet-200 on O(3)^d-rotated ImageNet (ablation) | ResNet family · ImageNet | top-1 accuracy gap rotated vs unrotated: about 10% to about 6% | Helped |
| Scale EfficientNet from B0 to B7 on O(3)^d-rotated ImageNet (ablation) | EfficientNet family · ImageNet | top-1 accuracy gap rotated vs unrotated: about 4% to only about 1.% | Helped |
| Replace Gaussian first-layer initialization with uniform initialization (ablation) | CNN on CIFAR-10 · CIFAR-10 | test accuracy | No clear effect |
Closest papers by meaning
- How DNNs break the Curse of Dimensionality: Compositionality and Symmetry Learning (ICLR 2025 · similarity 0.88)
- Symmetry Induces Structure and Constraint of Learning (ICML 2024 · similarity 0.87)
- On the hardness of learning under symmetries (ICLR 2024 · similarity 0.87)
- Symmetries in Overparametrized Neural Networks: A Mean Field View (NeurIPS 2024 · similarity 0.87)
- Improving Equivariant Networks with Probabilistic Symmetry Breaking (ICLR 2025 · similarity 0.86)
- Remove Symmetries to Control Model Expressivity and Improve Optimization (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.