Research map / Architectures and efficiency / Quantization
Overcoming Oscillations in Quantization-Aware Training
Markus Nagel, Marios Fournarakis, Yelysei Bondarenko, Tijmen Blankevoort
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Re-estimate batch-normalization statistics after training | MobileNetV2 QAT · ImageNet, 3-bit weights | validation accuracy (%): 64.97 -> 69.50 | Improved |
| Add oscillation dampening | MobileNetV2 LSQ · ImageNet, 4/4-bit weights/activations | validation accuracy (%): 69.5 -> 70.5 | Improved |
| Add iterative weight freezing | MobileNetV2 LSQ · ImageNet, 4/4-bit weights/activations | validation accuracy (%): 69.5 -> 70.6 | Improved |
| Add oscillation dampening | MobileNetV2 LSQ · ImageNet, 3/3-bit weights/activations | validation accuracy (%): 65.3 -> 67.8 | Improved |
| Add iterative weight freezing | MobileNetV2 LSQ · ImageNet, 3/3-bit weights/activations | validation accuracy (%): 65.3 -> 67.6 | Improved |
| Add oscillation dampening | MobileNetV3-Small LSQ · ImageNet, 3/3-bit weights/activations | validation accuracy (%): 52.0 -> 59.0 | Improved |
| Add iterative weight freezing | EfficientNet-Lite LSQ · ImageNet, 4/4-bit weights/activations | validation accuracy (%): 72.3 -> 73.5 | Improved |
| Use fixed, stronger dampening (λ=10^{-2}) rather than the baseline (ablation) | MobileNetV2 with 3-bit weights · ImageNet, post-BN | validation accuracy (%): 69.50 -> 68.57 | Hurt |
| Use a cosine-annealed dampening coefficient instead of no dampening (ablation) | MobileNetV2 with 3-bit weights · ImageNet, post-BN | validation accuracy (%): 69.50 -> 70.37 | Improved |
| Set a constant freezing threshold to 0.015 (ablation) | MobileNetV2 with 3-bit weights · ImageNet, post-BN | validation accuracy (%): 69.50 -> 70.13 | Improved |
| Anneal the freezing threshold from 0.04 to 0.01 (ablation) | MobileNetV2 with 3-bit weights · ImageNet, post-BN | validation accuracy (%): 69.50 -> 70.33 | Improved |
| Use too-low constant freezing threshold (0.01) (ablation) | MobileNetV2 with 3-bit weights · ImageNet, post-BN | validation accuracy (%): 69.50 -> 69.18 | Hurt |
Closest papers by meaning
- Oscillation-free Quantization for Low-bit Vision Transformers (ICML 2023 · similarity 0.92)
- Allowing Oscillation Quantization: Overcoming Solution Space Limitation in Low Bit-Width Quantization (ICCV 2025 · similarity 0.92)
- Solving Oscillation Problem in Post-Training Quantization Through a Theoretical Perspective (CVPR 2023 · similarity 0.90)
- Scheduling Weight Transitions for Quantization-Aware Training (ICCV 2025 · similarity 0.89)
- A2Q+: Improving Accumulator-Aware Weight Quantization (ICML 2024 · similarity 0.87)
- One-Step Forward and Backtrack: Overcoming Zig-Zagging in Loss-Aware Quantization Training (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
