Research map / Architectures and efficiency / Pruning and efficient tuning
Low-rank lottery tickets: finding efficient low-rank neural networks via matrix differential equations
Steffen Schotthöfer, Emanuele Zangrando, Jonas Kusch, Gianluca Ceruti, Francesco Tudisco
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train LeNet5 using adaptive low-rank factors at τ=0.11 | LeNet5 · MNIST | test accuracy: 99.2% -> 98.0% | Hurt |
| Increase the adaptive truncation threshold from τ=0.11 to τ=0.3 (ablation) | DLRT on LeNet5 · MNIST | evaluation parameter compression: 88.86% -> 96.4% | Improved |
| Increase the adaptive truncation threshold from τ=0.11 to τ=0.3 (ablation) | DLRT on LeNet5 · MNIST | test accuracy: 98.0% -> 95.3% | Hurt |
| Use DLRT rather than LeNet5 full-rank SGD training | LeNet5 · MNIST | evaluation parameter compression: 0% -> 96.4% | Improved |
| Use DLRT rather than full-rank training | ResNet-50 · ImageNet1k | top-5 test accuracy difference to baseline: -0.56 | Mixed |
| Use DLRT rather than full-rank training | VGG16 · ImageNet1k | top-5 test accuracy difference to baseline: -2.19 | Hurt |
| Use DLRT rather than full-rank training | VGG16 · CIFAR-10 | test accuracy difference to baseline: -1.89 | Hurt |
| Use DLRT rather than full-rank training | AlexNet · CIFAR-10 | test accuracy difference to baseline: -1.79 | Hurt |
| Truncate full-rank weight SVD without low-rank retraining (ablation) | 5-layer 784-neuron network · MNIST | test accuracy: 98.63% -> 9.91% | Hurt |
| Add low-rank retraining after truncating the SVD at rank 100 (ablation) | 5-layer 784-neuron network · MNIST | test accuracy: 9.91% -> 98.16% | Improved |
| Use fixed low-rank factors at ranks [5,5,5,5,10] instead of full-rank weights (ablation) | 5-layer 5120-neuron network · MNIST | batch training time: 0.320 -> 0.071 | Improved |
| Use fixed low-rank factors at ranks [5,5,5,5,10] instead of full-rank weights (ablation) | 5-layer 5120-neuron network · MNIST | dataset prediction time: 1.2476 -> 0.2472 | Improved |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 0 of 1 on most shared results.
- ThiNet — ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression (ICCV 2017) · better on 1 of 2 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Efficient Learning with Sine-Activated Low-Rank Matrices (ICLR 2025 · similarity 0.86)
- A geometric framework for momentum-based optimizers for low-rank training (NeurIPS 2025 · similarity 0.85)
- Inferring stochastic low-rank recurrent neural networks from neural data (NeurIPS 2024 · similarity 0.84)
- Maestro: Uncovering Low-Rank Structures via Trainable Decomposition (ICML 2024 · similarity 0.84)
- Flora: Low-Rank Adapters Are Secretly Gradient Compressors (ICML 2024 · similarity 0.84)
- COALA: Numerically Stable and Efficient Framework for Context-Aware Low-Rank Approximation (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
