Research map / Architectures and efficiency / Pruning and efficient tuning
Encoding Weights of Irregular Sparsity for Fixed-to-Fixed Model Compression
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add two shift registers for sequential encoding (ablation) | XOR-gate fixed-to-fixed compression · Transformer and ResNet-50 pruning settings | memory reduction: 50.3 -> 65.3; 82.4 -> 88.2 | Helped |
| Add one shift register instead of using non-sequential encoding (ablation) | XOR-gate fixed-to-fixed compression · Transformer WMT14 en-de, FP32, 70% magnitude pruning | memory reduction: 50.3 -> 63.1 | Helped |
| Add one shift register instead of using non-sequential encoding (ablation) | XOR-gate fixed-to-fixed compression · Transformer WMT14 en-de, FP32, 90% random pruning | memory reduction: 83.5 -> 88.5 | Helped |
| Increase shift-register count from one to two (ablation) | XOR-gate fixed-to-fixed compression · Transformer WMT14 en-de, FP32, 70% magnitude pruning | memory reduction: 63.1 -> 65.3 | Helped |
| Increase shift-register count from one to two (ablation) | XOR-gate fixed-to-fixed compression · 1M random bits, 90% sparsity, Nin=8 | memory reduction: 88.5% -> 89.3% | Helped |
| Invert binary weight vectors when unpruned bits contain fewer zeros than ones (ablation) | Non-sequential or Ns=1 encoding of FP32 weights · Transformer and ResNet-50 | encoding efficiency | Helped |
| Apply inverting at Ns=2 (ablation) | Sequential encoding · FP32 model weights | encoding efficiency | No clear effect |
| Use sequential input reuse rather than a non-sequential XOR decoder (ablation) | Fixed-to-fixed encoding on pruned neural-network weights · ResNet-50 ImageNet, 70% magnitude pruning | memory reduction: 52.2 -> 66.5 | Helped |
| Use sequential input reuse rather than a non-sequential XOR decoder (ablation) | Fixed-to-fixed encoding on pruned neural-network weights · ResNet-50 ImageNet, signed INT8, 90% random pruning | memory reduction: 83.3 -> 89.0 | Helped |
| Use magnitude-based or L0 regularization pruning rather than random pruning | Sequential encoding, Ns=2 · Transformer selected layers at target sparsity 0.7 | encoding efficiency: 99.8 -> 99.6 | Hurt |
Closest papers by meaning
- Layerwise Sparse Coding for Pruned Deep Neural Networks with Extreme Compression Ratio (AAAI 2020 · similarity 0.90)
- “Learning-Compression” Algorithms for Neural Net Pruning (CVPR 2018 · similarity 0.89)
- TETRIS: TilE-matching the TRemendous Irregular Sparsity (NeurIPS 2018 · similarity 0.89)
- DominoSearch: Find layer-wise fine-grained N:M sparse schemes from dense neural networks (NeurIPS 2021 · similarity 0.89)
- Targeted Low-rank Refinement: Enhancing Sparse Language Models with Precision (ICML 2025 · similarity 0.87)
- Sparsest Models Elude Pruning: An Exposé of Pruning’s Current Capabilities (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.