Research map / Architectures and efficiency / Quantization
Alternating Multi-bit Quantization for Recurrent Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use alternating multi-bit quantization instead of refined greedy quantization | LSTM · PTB test | PPW: 100.3 -> 95.8 (2/2); 95.6 -> 91.9 (2/3); 91.3 -> 87.9 (3/3) | Helped |
| Use alternating multi-bit quantization instead of refined greedy quantization | GRU · PTB test | PPW: 105.1 -> 101.2 (2/2); 100.3 -> 97.0 (2/3); 95.9 -> 92.9 (3/3) | Helped |
| Use 3-bit weights and activations rather than full precision | LSTM · PTB test | PPW: 89.8 -> 87.9 | Helped |
| Use alternating multi-bit quantization instead of refined greedy quantization | LSTM · WikiText-2 test | PPW: 108.7 -> 106.1 (2/2); 105.8 -> 102.7 (2/3); 102.2 -> 98.7 (3/3) | Helped |
| Use alternating multi-bit quantization instead of refined greedy quantization | LSTM · Text8 test | PPW: 135.6 -> 108.8 (2/2); 122.3 -> 105.1 (2/3); 110.2 -> 98.8 (3/3) | Helped |
| Use 2-bit weights and activations instead of full precision | RNNs · Language model inference | memory saving: 16 \times | Helped |
| Use 2-bit weights and activations instead of full precision | RNNs on CPUs · Language model inference | inference acceleration: 6 \times | Helped |
| Use 3-bit weights and activations instead of full precision | RNNs · Language model inference | memory saving: 10.5\times | Helped |
| Use 3-bit weights and activations instead of full precision | RNNs on CPUs · Language model inference | inference acceleration: 3 \times | Helped |
| Use alternating quantization instead of refined greedy quantization | LSTM · Sequential MNIST test | error rate: 1.39~\% -> 1.19~\% | Helped |
| Use alternating quantization instead of refined greedy quantization | MLP · MNIST test | error rate: 1.22\% -> 1.13\% | Helped |
| Use alternating quantization instead of refined greedy quantization | CNN · CIFAR-10 test | error rate: 12.08~\% -> 11.70~\% | Helped |
Closest papers by meaning
- HitNet: Hybrid Ternary Recurrent Neural Network (NeurIPS 2018 · similarity 0.89)
- Learning Recurrent Binary/Ternary Weights (ICLR 2019 · similarity 0.89)
- Adaptive Loss-Aware Quantization for Multi-Bit Networks (CVPR 2020 · similarity 0.87)
- And the Bit Goes Down: Revisiting the Quantization of Neural Networks (ICLR 2020 · similarity 0.87)
- QBB: Quantization with Binary Bases for LLMs (NeurIPS 2024 · similarity 0.86)
- Accelerating Linear Recurrent Neural Networks for the Edge with Unstructured Sparsity (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.