Research map / Architectures and efficiency / Pruning and efficient tuning
Stable Low-rank Tensor Decomposition for Compression of Convolutional Neural Network
Anh-Huy Phan, Konstantin Sobolev, Konstantin Sozykin, Dmitry Ermilov, Julia Gusak, Petr Tichavský, Valeriy Glukhov, Ivan Oseledets, Andrzej Cichocki
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add error-preserving sensitivity correction (EPC) to CPD of kernels (ablation) | CPD-compressed ResNet-18 · ILSVRC-12, ResNet-18 layers | top-1 accuracy: ~2% top-1 difference on some deep layers | Improved |
| add EPC correction to CPD at rank 500 (ablation) | CPD-compressed ResNet-18 layer4.1.conv1 · ILSVRC-12 | top-1 accuracy: rank 450 -> original accuracy; rank 250 -> less than 1% loss | Improved |
| add EPC correction to CPD (ablation) | CPD-compressed ResNet-18 · ILSVRC-12 and CIFAR-100 | sensitivity of decomposition | Improved |
| replace CPD-EPC with TKD-CPD-EPC hybrid at small ranks | ResNet-18 layer 4.0.conv1 · CIFAR-100 | top-1 accuracy | Hurt |
| replace CPD-EPC with TKD-CPD-EPC hybrid at CP rank >= 110 | ResNet-18 layer 4.0.conv1 · CIFAR-100 | top-1/top-5 accuracy: R_CP >= 110 | Improved |
| replace CPD-EPC with TKD-CPD-EPC | ResNet-18 single layers · CIFAR-100 | accuracy vs FLOPs/parameters | Improved |
| compress ResNet-50 whole network with ours | ResNet-50 · ILSVRC-12 | inference time on CPU: 3.92 -> 2.84 | Improved |
| compress ResNet-50 whole network with ours | ResNet-50 · ILSVRC-12 | inference time on V100 GPU: 102.3 -> 89.5 | Improved |
| compress ResNet-50 whole network with ours | ResNet-50 · ILSVRC-12 | inference time on Snapdragon 845: 221 -> 171 | Improved |
| our method vs MUSCO on ResNet-18 | ResNet-18 · ILSVRC-12 | FLOPs reduction and top-1 drop: 2.42 -> 3.09 FLOPs; -0.47 -> -0.69 top-1 | Mixed |
Closest papers by meaning
- Distribution-Aware Tensor Decomposition for Compression of Convolutional Neural Networks (NeurIPS 2025 · similarity 0.92)
- Exploring Unexplored Tensor Network Decompositions for Convolutional Neural Networks (NeurIPS 2019 · similarity 0.91)
- Towards Efficient Tensor Decomposition-Based DNN Model Compression With Optimization Framework (CVPR 2021 · similarity 0.89)
- Beyond Lazy Training for Over-parameterized Tensor Decomposition (NeurIPS 2020 · similarity 0.88)
- How Informative is the Approximation Error from Tensor Decomposition for Neural Network Compression? (ICLR 2023 · similarity 0.88)
- Convolutions and More as Einsum: A Tensor Network Perspective with Advances for Second-Order Methods (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
