Research map / Architectures and efficiency / Pruning and efficient tuning
Minimal Random Code Learning: Getting Bits Back from Compressed Model Parameters
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MIRACLE to encode random weight samples from a variational distribution instead of deterministically encoding a weight set | LeNet-5 · MNIST | test error at a comparable compressed model size | Helped |
| Use MIRACLE to encode random weight samples from a variational distribution instead of deterministically encoding a weight set | VGG-16 · CIFAR-10 | size at comparable test performance | Helped |
| Add the hashing trick to restrict the optimization space and reduce the dimensions of the encoding and variational distributions (ablation) | MIRACLE · LeNet-5 on MNIST and VGG-16 on CIFAR-10 | compression rate: ∼ 1.5 × | Helped |
Closest papers by meaning
- Compressing Neural Networks: Towards Determining the Optimal Layer-wise Decomposition (NeurIPS 2021 · similarity 0.89)
- Layerwise Sparse Coding for Pruned Deep Neural Networks with Extreme Compression Ratio (AAAI 2020 · similarity 0.89)
- Soft Weight-Sharing for Neural Network Compression (ICLR 2017 · similarity 0.89)
- The Description Length of Deep Learning models (NeurIPS 2018 · similarity 0.89)
- Model Preserving Compression for Neural Networks (NeurIPS 2022 · similarity 0.88)
- "Lossless" Compression of Deep Neural Networks: A High-dimensional Neural Tangent Kernel Approach (NeurIPS 2022 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.