Research map / Architectures and efficiency / Pruning and efficient tuning
How Informative is the Approximation Error from Tensor Decomposition for Neural Network Compression?
Jetze Schuurmans, kim batselier, Julian Kooij
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use relative weight approximation error to choose among tensor-decomposition options | TD-compressed GaripovNet and ResNet-18 · CIFAR-10; Fashion-MNIST | Kendall's tau correlation with classification error | Improved |
| Measure approximation error on output features instead of weights (ablation) | TD-compressed CNNs · CIFAR-10; Fashion-MNIST | Kendall's tau correlation with classification error | No clear effect |
| Use relative rather than absolute weight error (ablation) | TD-compressed ResNet-18 · CIFAR-10 | Kendall's tau correlation with classification error | Improved |
| Compare choices at higher rather than lower compression levels (ablation) | TD-compressed GaripovNet and ResNet-18 · CIFAR-10; Fashion-MNIST | Kendall's tau correlation with classification error | Improved |
| Compare choices at only 10% or 25% compression (ablation) | TD-compressed GaripovNet and ResNet-18 · CIFAR-10; Fashion-MNIST | Kendall's tau correlation with classification error | Hurt |
| Fine-tune the decomposed model after compression (ablation) | TD-compressed GaripovNet · CIFAR-10; Fashion-MNIST | Kendall's tau correlation with performance error | Hurt |
| Fine-tune the decomposed model after compression (ablation) | TD-compressed ResNet-18 · CIFAR-10 | Kendall's tau correlation with performance error | No clear effect |
| Compare layers while fixing decomposition method and compression level (ablation) | TD-compressed CNNs · CIFAR-10; Fashion-MNIST | Kendall's tau correlation with performance error | Hurt |
| Compare decomposition methods for a fixed layer (ablation) | TD-compressed CNNs · CIFAR-10; Fashion-MNIST | Kendall's tau correlation with performance error | Improved |
| Fine-tune after comparing decomposition methods (ablation) | TD-compressed GaripovNet · CIFAR-10; Fashion-MNIST | Kendall's tau correlation with performance error | Hurt |
Closest papers by meaning
- Distribution-Aware Tensor Decomposition for Compression of Convolutional Neural Networks (NeurIPS 2025 · similarity 0.89)
- Stable Low-rank Tensor Decomposition for Compression of Convolutional Neural Network (ECCV 2020 · similarity 0.88)
- Compressing Neural Networks: Towards Determining the Optimal Layer-wise Decomposition (NeurIPS 2021 · similarity 0.87)
- Approximation to Smooth Functions by Low-Rank Swish Networks (ICML 2025 · similarity 0.85)
- COALA: Numerically Stable and Efficient Framework for Context-Aware Low-Rank Approximation (NeurIPS 2025 · similarity 0.85)
- Implicit Compressibility of Overparametrized Neural Networks Trained with Heavy-Tailed SGD (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
