Research map / Architectures and efficiency / Quantization
Divide and Conquer: Leveraging Intermediate Feature Representations for Quantized Training of Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add DCQ sectional intermediate-feature distillation to DoReFa-Net with binary weights | DoReFa-Net · Various networks (average) | accuracy: 21.6% | Helped |
| Add DCQ sectional intermediate-feature distillation to DoReFa-Net with ternary weights | DoReFa-Net · Various networks (average) | accuracy: 9.3% | Helped |
| Use DCQ with W4/A4 quantization | Quantized training · AlexNet, ResNet-18, MobileNet | accuracy: 0.98% | Helped |
| Use DCQ with W3/A3 quantization | Quantized training · AlexNet, ResNet-18, MobileNet | accuracy: 0.96% | Helped |
| Train intermediate features with MSE rather than the other examined regression losses (ablation) | DCQ · AlexNet binary quantization; trends reported for other networks | intermediate feature-map loss | Helped |
| Increase the split count from single-stage to two-stage and four-stage training (ablation) | VGG-11 ternary quantization · VGG-11 | convergence and final accuracy | Helped |
| Train binary kernels with DCQ instead of directly binarizing full-precision kernels (ablation) | LeNet · LeNet | accuracy: 10.8% vs 98.1% | Helped |
| Train binary kernels with DCQ instead of directly binarizing full-precision kernels (ablation) | AlexNet · AlexNet | accuracy: 40.5% and 55.6% | Helped |
| Use DCQ rather than conventional end-to-end quantized training (ablation) | Binary quantized networks · AlexNet, ResNet-20, VGG-11 | amount of binary weights altered during training | Helped |
| Use DCQ rather than conventional end-to-end quantized training (ablation) | Quantized networks · Various networks | convergence and solution quality | Helped |
| Add DCQ to Apprentice-style quantization | ResNet-20 and ResNet-18 · CIFAR10 and ImageNet | accuracy | Helped |
Closest papers by meaning
- Model compression via distillation and quantization (ICLR 2018 · similarity 0.89)
- Deep Neural Network Quantization via Layer-Wise Optimization Using Limited Training Data (AAAI 2019 · similarity 0.89)
- Implicit Feature Decoupling With Depthwise Quantization (CVPR 2022 · similarity 0.88)
- Training Quantized Nets: A Deeper Understanding (NeurIPS 2017 · similarity 0.88)
- Automatic Joint Structured Pruning and Quantization for Efficient Neural Network Training and Compression (CVPR 2025 · similarity 0.87)
- Learning to Quantize for Training Vector-Quantized Networks (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.