Research map / Architectures and efficiency / Quantization
Mixed Precision DNNs: All you need is a good parametrization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| parametrize uniform quantizer with stepsize d and dynamic range qmax (U3) instead of b,d or b,qmax (ablation) | ResNet-20 uniform quantization · CIFAR-10 | validation error: 7.32% vs 8.50% (U1) and 8.50% (U2) for weights-only, float init | Helped |
| parametrize power-of-two quantizer with qmin and qmax (P3) instead of b-based parametrizations (ablation) | ResNet-20 power-of-two quantization · CIFAR-10 weights+activations | validation error: 9.86% vs 11.68% (P2, b,qmin) and 15.10% vs 22.91% etc. (float init) | Helped |
| learn bitwidth directly as a parameter (b-based parametrization) (ablation) | uniform and power-of-two quantizers · CIFAR-10 ResNet-20 | validation error: e.g. 17.8% vs 8.18% for b,qmax weights-only random init; diverging for P1 weights+activations | Hurt |
| use Adam instead of SGD for uniform parametrization comparison (ablation) | ResNet-20 quantization parametrizations · CIFAR-10 | error rate: U1 11.74 (SGD) vs 7.61 (Adam); U3 7.32 (SGD) vs 7.36 (Adam) | Mixed |
| learned stepsize/dynamic range with memory constraints (8a),(8c) vs fixed 4bit quantization | MobileNetV2 and ResNet-18 4bit on ImageNet · ImageNet | validation error: 30.26% vs 36.27% (MobileNetV2); 29.92% vs 34.15% (ResNet-18) | Helped |
| learned quantization parameters (TQT-style dynamic range) vs proposed parametrization with memory constraints | MobileNetV2 and ResNet-18 4bit on ImageNet · ImageNet | validation error: 30.26% vs 32.21% (MobileNetV2); 29.92% vs 30.49% (ResNet-18) | Helped |
| remove memory constraints during training (ablation) | MobileNetV2 and ResNet-18 ImageNet · ImageNet | validation error: 29.41% vs 30.26% (MobileNetV2); 29.34% vs 29.92% (ResNet-18), but at much larger size | No clear effect |
| learn qmax with bitwidth fixed at 2 bit (TQT) and learn both qmax and b under 70KB constraint (Ours) on CIFAR-10 weights-only (ablation) | ResNet-20 weight quantization · CIFAR-10 | validation error: 8.59% (Ours) vs 9.47% (TQT) vs 10.81% (Fixed) for uniform | Helped |
Closest papers by meaning
- Not All Bits have Equal Value: Heterogeneous Precisions via Trainable Noise (NeurIPS 2022 · similarity 0.88)
- InfoQ: Mixed-Precision Quantization via Global Information Flow (AAAI 2026 · similarity 0.88)
- Training Quantized Neural Networks With a Full-Precision Auxiliary Module (CVPR 2020 · similarity 0.88)
- Towards Cheaper Inference in Deep Networks with Lower Bit-Width Accumulators (ICLR 2024 · similarity 0.87)
- FracBits: Mixed Precision Quantization via Fractional Bit-Widths (AAAI 2021 · similarity 0.87)
- Learning from Loss Landscape: Generalizable Mixed-Precision Quantization via Adaptive Sharpness-Aware Gradient Aligning (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.