Research map / Architectures and efficiency / Quantization
CAT: Compression-Aware Training for bandwidth reduction
Chaim Baskin, Brian Chmiel, Evgenii Zheltonozhskii, Ron Banner, Alex M. Bronstein, Avi Mendelson
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add entropy penalty (lambda 0.05) during training (ablation) | CAT · ImageNet val (ResNet-18, 5-bit) | memory bits per value: 3.100 -> 1.930 (6-bit compute, lambda 0 vs 0.05) | Improved |
| increase lambda from 0.05 to 0.3 (ablation) | CAT · ImageNet val (ResNet-18, 5-bit) | top-1 accuracy (%): 69.490 -> 67.700 | Hurt |
| replace soft entropy with compressibility loss (ablation) | CAT · ImageNet val (ResNet-18) | top-1 accuracy (%): 67.86% -> 67.84% (4-bit); 69.49% -> 69.36% (5-bit) | No clear effect |
| increase batch size from 16 to 64 (ablation) | CAT · ImageNet val (ResNet-50, 4-bit) | top-1 accuracy (%): 73.800 -> 74.5 | Improved |
| exponential temperature scheduling for soft entropy (ablation) | CAT · ImageNet val | top-1 accuracy (%) | No clear effect |
| vary random seed over multiple runs (ablation) | CAT · ImageNet val (ResNet-18, 5-bit) | top-1 accuracy (%): 69.122±0.016 over 5 runs | No clear effect |
| CAT versus prior detection compression on SSD512 | CAT · PASCAL VOC detection | accuracy (%): 68.12 -> 64.39 (prior, 8-bit); 80.72 -> 77.49 (CAT, 6-bit) | Improved |
| add CAT 5-bit compute to ResNet-18 | CAT · ImageNet val | compression ratio: 1 -> 3.33; accuracy 69.70 -> 69.20 | Improved |
Closest papers by meaning
- Focused Quantization for Sparse CNNs (NeurIPS 2019 · similarity 0.89)
- Coreset-Based Neural Network Compression (ECCV 2018 · similarity 0.89)
- Transform Quantization for CNN Compression (TPAMI 2021 · similarity 0.88)
- Permute, Quantize, and Fine-Tune: Efficient Compression of Neural Networks (CVPR 2021 · similarity 0.88)
- Network Memory Footprint Compression Through Jointly Learnable Codebooks and Mappings (ICLR 2024 · similarity 0.88)
- Distribution-Aware Tensor Decomposition for Compression of Convolutional Neural Networks (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
