Research map / Architectures and efficiency / CNN design
The Singular Values of Convolutional Layers
Hanie Sedghi, Vineet Gupta, Philip M. Long
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add exact operator-norm projection with clip value 0.5 every 100 steps | 32-layer ResNet · CIFAR-10 | test error: 6.2% -> 5.3% | Improved |
| Use clip value 1.0 rather than 0.5 (ablation) | Operator-norm projection on 32-layer ResNet · CIFAR-10 | test error: 5.3% -> 5.5% | Hurt |
| Use clip value 0.1 rather than 0.5 (ablation) | Operator-norm projection on 32-layer ResNet · CIFAR-10 | test error: 5.3% -> 6.7% | Hurt |
| Use clip values 2.5 or 3.0 rather than 0.5 (ablation) | Operator-norm projection on 32-layer ResNet · CIFAR-10 | test error | No clear effect |
| Remove batch normalization while testing operator-norm regularization (ablation) | 32-layer ResNet · CIFAR-10 | test error | Mixed |
| Replace exact convolution-operator clipping with reshaped-K clipping every 100 steps (ablation) | 32-layer ResNet · CIFAR-10 | test accuracy | No clear effect |
| Clip reshaped-K singular values every step instead of every 100 steps (ablation) | Reshaped-K clipping · CIFAR-10 | test accuracy | Hurt |
| Use exact operator-norm clipping instead of reshaped-K clipping | 32-layer ResNet · CIFAR-10, 10000 batches | training time: 14490 seconds -> 11004 seconds | Improved |
Closest papers by meaning
- Towards Practical Control of Singular Values of Convolutional Layers (NeurIPS 2022 · similarity 0.88)
- Convolutional Normalization: Improving Deep Convolutional Network Robustness and Training (NeurIPS 2021 · similarity 0.86)
- Revealing the Structure of Deep Neural Networks via Convex Duality (ICML 2021 · similarity 0.85)
- Fantastic Four: Differentiable and Efficient Bounds on Singular Values of Convolution Layers (ICLR 2021 · similarity 0.85)
- LayerAct: Advanced Activation Mechanism for Robust Inference of CNNs (AAAI 2025 · similarity 0.84)
- Towards Training Without Depth Limits: Batch Normalization Without Gradient Explosion (ICLR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
