Research map / Architectures and efficiency / CNN design
Towards Practical Control of Singular Values of Convolutional Layers
Alexandra Senderovich, Ekaterina Bulatova, Anton Obukhov, Maxim Rakhuba
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add TT decomposition rank 256 to SOC | LipConvNet-5 · CIFAR-10 | accuracy: 75.6 ± 0.3 -> 78.3 ± 0.1 | Improved |
| add TT decomposition rank 256 to SOC | LipConvNet-5 · CIFAR-10 AutoAttack | robust accuracy: 31.1 ± 0.2 -> 34.7 ± 0.6 | Improved |
| add TT decomposition rank 256 to SOC | LipConvNet-20 · CIFAR-10 | accuracy: 76.3 ± 0.5 -> 78.4 ± 0.2 | Improved |
| use TT rank 128 instead of 256 in SOC-TT | LipConvNet-30 · CIFAR-10 | accuracy: 76.3 ± 1.0 -> 76.0 ± 0.1 | No clear effect |
| add TT decomposition rank 128 to SOC | LipConvNet-5 · CIFAR-10 | speedup: 1.0 -> 1.4 | Improved |
| add TT decomposition rank 256 to SOC | LipConvNet-5 · CIFAR-10 | speedup: 1.0 -> 0.9 | Hurt |
| increase orthogonal loss weight from 5e3 to 5e4 (ablation) | SOC-TT rank 256 · CIFAR-10 LipConvNet-5 | accuracy: 78.37 -> 75.74 | Hurt |
| increase orthogonal loss weight from 5e3 to 5e4 (ablation) | SOC-TT rank 256 · CIFAR-10 LipConvNet-5 | Lipschitz constant: 0.93 -> 0.58 | Improved |
| clip singular values to 2 instead of 1 | WRN-16-10 · CIFAR-10 | accuracy: 95.27 -> 95.99 | Improved |
| add TT rank 256 to clipping to 1 | WRN-16-10 · CIFAR-10 | accuracy: 95.27 -> 95.12 | No clear effect |
| add TT rank 256 to division | WRN-16-10 · CIFAR-10 AutoAttack | robust accuracy: 53.39 -> 56.28 | Improved |
| add TT rank 320 to division | WRN-16-10 · CIFAR-10 | accuracy: 95.17 -> 94.93 | Hurt |
Closest papers by meaning
- The Singular Values of Convolutional Layers (ICLR 2019 · similarity 0.88)
- Exploring Unexplored Tensor Network Decompositions for Convolutional Neural Networks (NeurIPS 2019 · similarity 0.86)
- CNN-Cert: An Efficient Framework for Certifying Robustness of Convolutional Neural Networks (AAAI 2019 · similarity 0.86)
- The Unreasonable Effectiveness of Fully-Connected Layers for Low-Data Regimes (NeurIPS 2022 · similarity 0.86)
- Convolutions and More as Einsum: A Tensor Network Perspective with Advances for Second-Order Methods (NeurIPS 2024 · similarity 0.84)
- Distribution-Aware Tensor Decomposition for Compression of Convolutional Neural Networks (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
