Research map / Architectures and efficiency / Quantization
Structured Compression by Weight Encryption for Unstructured Pruning and Quantization
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply the proposed lossless XOR-gate weight representation to pruned, quantized weights | LeNet5 FC1 · MNIST | bits per weight: 0.19 bits per weight; 11× smaller than ternary quantization | Helped |
| Apply the proposed XOR-gate weight representation | AlexNet FC5 and FC6 · ImageNet | bits per weight: 0.28 bits per weight; ternary quantization requires 2 bits per weight | Helped |
| Apply the proposed SQNN format after pruning and quantization | ResNet32 · CIFAR10 | bits per weight: 1.22 bits per weight; 3 bits would be required without our proposed compression techniques | Helped |
| Apply the proposed representation to pruned, 2-bit-quantized weights | LSTM · PTB | bits per weight: 1.67 bits per weight | Helped |
| Prune and quantize weights before applying the proposed representation | AlexNet · ImageNet | Top-1 accuracy: 57.6% (T1) -> 55.9% (T1) | Hurt |
| Prune and quantize weights before applying the proposed representation | ResNet32 · CIFAR10 | accuracy: 92.5% -> 91.6% | Hurt |
| Prune and quantize weights before applying the proposed representation | LSTM · PTB | perplexity per word: 89.6 PPW -> 93.9 PPW | Hurt |
| Use a larger XOR seed input size | Synthetic random weight matrices · Synthetic data | memory reduction | Helped |
| Increase the pruning rate | Synthetic random weight matrices · Synthetic data | memory reduction: ≈ 0.83 | Helped |
Closest papers by meaning
- Layerwise Sparse Coding for Pruned Deep Neural Networks with Extreme Compression Ratio (AAAI 2020 · similarity 0.90)
- Double Viterbi: Weight Encoding for High Compression Ratio and Fast On-Chip Reconstruction for Deep Neural Network (ICLR 2019 · similarity 0.90)
- Harmonious Coexistence of Structured Weight Pruning and Ternarization for Deep Neural Networks (AAAI 2020 · similarity 0.89)
- CLIP-Q: Deep Network Compression Learning by In-Parallel Pruning-Quantization (CVPR 2018 · similarity 0.89)
- Encoding Weights of Irregular Sparsity for Fixed-to-Fixed Model Compression (ICLR 2022 · similarity 0.89)
- SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.