Research map / Architectures and efficiency / RNNs and normalization
Low-Rank Compression of Neural Nets: Learning the Rank of Each Layer
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Jointly learn each layer's rank and weights instead of choosing ranks with heuristics | LeNet300 · MNIST | test error (%): 1.98 -> 1.87 | Helped |
| Increase the rank-selection cost weight from λ=1 × 10−3 to λ=2 × 10−3 | LeNet300 · MNIST | ρFLOPs: 5.87 -> 8.59 | Helped |
| Use learned low ranks rather than the reference architecture | LeNet5 · MNIST | test error (%): 0.55 -> 0.75 | Hurt |
| Increase the rank-selection cost weight from λ=0.4 × 10−3 to λ=1.1 × 10−3 | LeNet5 · MNIST | ρFLOPs: 6.98 -> 11.49 | Helped |
| Learn ranks with scheme 1 | AlexNet · ImageNet ILSVRC 2012 validation | top-1 error (%): 42.49 -> 41.56 | Helped |
| Use convolutional low-rank scheme 2 rather than scheme 1 | AlexNet with learned ranks · ImageNet ILSVRC 2012 validation | MFLOPs: 240 -> 151 | Helped |
| Use convolutional low-rank scheme 2 rather than scheme 1 | AlexNet with learned ranks · ImageNet ILSVRC 2012 validation | top-1 error (%): 41.56 (scheme 2) vs 41.56 (scheme 1); 42.69 vs 42.83 at listed settings | Mixed |
| Replace learned rank selection with fixed-rank low-rank compression | CIFAR10 networks · CIFAR10 | test error | Hurt |
| Use LC rank-and-weight optimization rather than COBLA | NIN · CIFAR10 | test error | Helped |
| Use learned rank selection instead of the published low-rank and filter-pruning alternatives | AlexNet · ImageNet ILSVRC 2012 validation | top-1 error (%): 42.83 vs 43.40; 42.69 vs 43.40 | Helped |
Closest papers by meaning
- Approximation to Smooth Functions by Low-Rank Swish Networks (ICML 2025 · similarity 0.88)
- Dynamical Low-Rank Compression of Neural Networks with Robustness under Adversarial Attacks (NeurIPS 2025 · similarity 0.87)
- Maestro: Uncovering Low-Rank Structures via Trainable Decomposition (ICML 2024 · similarity 0.87)
- Distribution-Aware Tensor Decomposition for Compression of Convolutional Neural Networks (NeurIPS 2025 · similarity 0.87)
- Low Rank Gradients and Where to Find Them (NeurIPS 2025 · similarity 0.87)
- COALA: Numerically Stable and Efficient Framework for Context-Aware Low-Rank Approximation (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.