Research map / Architectures and efficiency / Quantization
MetaQuant: Learning to Quantize by Learning to Penetrate Non-differentiable Quantization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace STE with learned meta quantizer (FCGrad) for gradient penetration, SGD | dorefa 1-bit quantization · CIFAR10 ResNet20 | test accuracy (%): 80.745(2.113) -> 88.840(0.291) | Helped |
| replace STE with learned meta quantizer (FCGrad) for gradient penetration, Adam | dorefa 1-bit quantization · CIFAR10 ResNet20 | test accuracy (%): 89.782(0.172) -> 89.962(0.068) | Helped |
| replace STE with meta quantizer (FCGrad), SGD | BWN 1-bit quantization · CIFAR10 ResNet20 | test accuracy (%): 75.913(3.495) -> 88.949(0.231) | Helped |
| replace STE with meta quantizer (FCGrad), SGD | dorefa on ResNet56 and ResNet110 · CIFAR100 | test accuracy (%): 42.265(8.143) -> 64.351(0.935) (ResNet56) | Helped |
| replace STE with meta quantizer (FCGrad), Adam | BWN ResNet110 · CIFAR100 | test accuracy (%): 66.265(1.429) -> 69.114(0.181) | Helped |
| replace STE with meta quantizer (FCGrad), Adam | dorefa ResNet18 · ImageNet top-1 | top-1 accuracy (%): 58.349(2.072) -> 59.835(0.359) | Helped |
| replace STE with meta quantizer (FCGrad), Adam | BWN ResNet18 · ImageNet top-1 | top-1 accuracy (%): 59.503(0.835) -> 60.328(0.391) | Helped |
| use MultiFC instead of LSTMFC/FCGrad meta quantizer architecture, SGD (ablation) | MetaQuant dorefa ResNet20 · CIFAR10 | test accuracy (%): 88.942(0.466) vs 88.305(0.810) vs 88.840(0.291) | No clear effect |
| use MetaQuant instead of STE under SGD vs Adam | quantization training · CIFAR10 ResNet20 | test accuracy (%) | Mixed |
| add meta quantizer during training (extra training cost) | dorefa STE ResNet20 · CIFAR10 | training time per iteration (s): 38.17 -> 51.15 | Hurt |
| MetaQuant vs ProxQuant (non-STE) accuracy drop | ResNet44 CIFAR10 · CIFAR10 accuracy drop | accuracy drop (%): 0.99 -> 0.08 | Helped |
Closest papers by meaning
- Learning to Quantize for Training Vector-Quantized Networks (ICML 2025 · similarity 0.90)
- Deep Neural Network Quantization via Layer-Wise Optimization Using Limited Training Data (AAAI 2019 · similarity 0.88)
- Accurate Neural Training with 4-bit Matrix Multiplications at Standard Formats (ICLR 2023 · similarity 0.88)
- Network Quantization With Element-Wise Gradient Scaling (CVPR 2021 · similarity 0.88)
- SYQ: Learning Symmetric Quantization for Efficient Deep Neural Networks (CVPR 2018 · similarity 0.87)
- SpinQuant: LLM Quantization with Learned Rotations (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.