Research map / Architectures and efficiency / Quantization
Bit-Mixer: Mixed-Precision Networks With Runtime Bit-Width Selection
Adrian Bulat, Georgios Tzimiropoulos
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Transitional Batch-Norm layers (ablation) | Bit-Mixer ResNet-18 · ImageNet random layer-wise bits | top-1 accuracy (%): 8.8 -> 65.8 | Improved |
| add Transitional Batch-Norm layers (ablation) | Bit-Mixer ResNet-18 · ImageNet 4-bit | top-1 accuracy (%): 8.2 -> 69.2 | Improved |
| add Transitional Batch-Norm layers (ablation) | Bit-Mixer ResNet-18 · ImageNet 2-bit | top-1 accuracy (%): 10.2 -> 64.4 | Improved |
| add FP32 teacher distillation to Stage III (ablation) | Bit-Mixer ResNet-18 · ImageNet 2-bit | top-1 accuracy (%): 64.0 -> 64.4 | Improved |
| add Stage I teacher distillation to Stage II (ablation) | Bit-Mixer ResNet-18 · ImageNet 2-bit | top-1 accuracy (%): 65.1 -> 65.6 | Improved |
| add Stage I teacher distillation to Stage III (ablation) | Bit-Mixer ResNet-18 · ImageNet random layer-wise bits | top-1 accuracy (%): 65.5 -> 65.7 | Improved |
| add layer-wise random bit training (Stage III) after Stage II | Bit-Mixer ResNet-18 · ImageNet 2-bit | top-1 accuracy (%): 65.6 -> 64.4 | Hurt |
| include 1-bit in the shared 4-3-2-1 meta-network | Bit-Mixer ResNet-18 · ImageNet 1-bit | top-1 accuracy (%): 59.0 -> 57.3 | Hurt |
| run Stage III with binary-ternary 1.5-bit weights | Bit-Mixer ResNet-18 · ImageNet 1.5-bit | top-1 accuracy (%): 64.0 -> 62.1 | Hurt |
| use asymmetric instead of symmetric quantization (ablation) | Bit-Mixer ResNet-18 · ImageNet 2-bit | top-1 accuracy (%): 65.1 -> 65.2 | No clear effect |
| use per bit-width scaling factors instead of shared scale-based quantization (ablation) | Bit-Mixer ResNet-18 · ImageNet | top-1 accuracy (%): 5--10% worse | Hurt |
| sample random sub-paths instead of progressive Stage III training (ablation) | Bit-Mixer ResNet-18 · ImageNet | top-1 accuracy (%) | Hurt |
Closest papers by meaning
- One-Shot Model for Mixed-Precision Quantization (CVPR 2023 · similarity 0.89)
- MetaMix: Meta-State Precision Searcher for Mixed-Precision Activation Quantization (AAAI 2024 · similarity 0.89)
- Retraining-Free Model Quantization via One-Shot Weight-Coupling Learning (CVPR 2024 · similarity 0.87)
- ScaleBITS: Scalable Bitwidth Search for Hardware-Aligned Mixed-Precision LLMs (NeurIPS 2026 · similarity 0.87)
- PTMQ: Post-training Multi-Bit Quantization of Neural Networks (AAAI 2024 · similarity 0.87)
- Towards Cheaper Inference in Deep Networks with Lower Bit-Width Accumulators (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
