Research map / Learning theory and algorithms / Kernels and regression theory
Tight and Efficient Upper Bound on Spectral Norm of Convolutional Layers
Ekaterina Grishina, Mikhail Gorbunov, Maxim Rakhuba
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use TN bound (tensor spectral norm) instead of F4 bound | spectral norm upper bound · Gaussian kernels, 3x3 to 7x7 | bound / exact ratio: F4 1.646-2.647 vs TN 1.008-1.131 of exact norm | Improved |
| add TN spectral norm regularization | ResNet18 · CIFAR100 (with weight decay) | top-1 accuracy: 73.84 -> 74.99 | Improved |
| add TN spectral norm regularization | ResNet18 · CIFAR100 (w/o weight decay) | top-1 accuracy: 73.10 -> 73.96 | Improved |
| add TN spectral norm regularization | ResNet34 · ImageNet | Acc@1: 73.368 -> 73.510 | No clear effect |
| replace F4 with TN as the regularized bound | ResNet18 · CIFAR100 (with weight decay) | top-1 accuracy: 74.91 -> 74.99 | No clear effect |
| TN regularization instead of LipBound for training cost | ResNet18 · CIFAR100 time per epoch | time per epoch (s): 242.5 -> 143.6 | Improved |
| add 2norm orthogonal regularizer on convolution kernel | ResNet18 · CIFAR100 | top-1 accuracy: 73.84 -> 75.99 | Improved |
| add Ratio regularizer | ResNet18 · CIFAR100 | top-1 accuracy: 73.84 -> 74.98 | Improved |
| increase 2norm coefficient from 5e-3 to 1e-2 (ablation) | 2norm regularizer · CIFAR100 | top-1 accuracy: 75.99 -> 74.43 | Hurt |
| 2norm vs OCNN orthogonal regularizer in cost | ResNet18 · CIFAR100 | time per epoch (s): 150.1 -> 171.8 | Hurt |
Closest papers by meaning
- Fantastic Four: Differentiable and Efficient Bounds on Singular Values of Convolution Layers (ICLR 2021 · similarity 0.94)
- The Sample Complexity of One-Hidden-Layer Neural Networks (NeurIPS 2022 · similarity 0.88)
- Efficient Bound of Lipschitz Constant for Convolutional Layers by Gram Iteration (ICML 2023 · similarity 0.88)
- Rethinking 3D Convolution in $\ell_p$-norm Space (NeurIPS 2024 · similarity 0.85)
- Distribution-Aware Tensor Decomposition for Compression of Convolutional Neural Networks (NeurIPS 2025 · similarity 0.84)
- Optimal Rates for Vector-Valued Spectral Regularization Learning Algorithms (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
