Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning
It's All in the Teacher: Zero-Shot Quantization Brought Closer to the Teacher
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace GDFQ's CE+KL objective with KL-only loss and add gradient inundation | GDFQ · CIFAR-100, ResNet-20, 4w4a | top-1 accuracy: 63.39 -> 65.80 | Helped |
| Apply AIT to GDFQ | GDFQ · ImageNet, ResNet-50, 4w4a | top-1 accuracy: +12.12 | Helped |
| Apply AIT to Qimera | Qimera · CIFAR-10, ResNet-20, 4w4a | top-1 accuracy: 91.26 -> 91.23 ( $-$0.03 ) | No clear effect |
| Apply AIT to Qimera | Qimera · ImageNet, across ResNet-18, ResNet-50, and MobileNetV2 | top-1 accuracy: +2.99; +1.38; +5.19 | Helped |
| Remove cross-entropy from GDFQ's loss without adding gradient inundation (ablation) | GDFQ · ImageNet, ResNet-50, 4w4a | top-1 accuracy: 52.12 -> 42.64 | Hurt |
| Add gradient inundation to KL-only GDFQ (ablation) | GDFQ KL-only · ImageNet, ResNet-50, 4w4a | top-1 accuracy: 42.64 -> 64.24 | Helped |
| Retain CE+KL and add gradient inundation (ablation) | GDFQ · ImageNet, ResNet-18, 4w4a | top-1 accuracy: 60.60 -> 55.01 | Hurt |
| Use CE-only loss with gradient inundation (ablation) | GDFQ · ImageNet, ResNet-50, 4w4a | top-1 accuracy: 52.12 -> 27.86 | Hurt |
| Increase the KL-only learning rate by ×100 instead of using gradient inundation (ablation) | GDFQ KL-only · ImageNet, ResNet-50, 4w4a | top-1 accuracy: 42.64 -> 61.68 | Helped |
| Increase the KL-only learning rate by ×100 instead of using gradient inundation (ablation) | GDFQ KL-only · CIFAR-10, ResNet-20, 4w4a | top-1 accuracy: 90.06 -> 92.20 | Helped |
| Vary gradient-inundation target proportion ρ (ablation) | AIT on GDFQ · CIFAR-100 ResNet-20 and ImageNet ResNet-18, 4w4a | top-1 accuracy | No clear effect |
| Replace SGD with Adam while retaining gradient inundation (ablation) | AIT · ImageNet, ResNet-50, 4w4a | top-1 accuracy: 52.12 -> 65.67 | Helped |
Closest papers by meaning
- Genie: Show Me the Data for Quantization (CVPR 2023 · similarity 0.93)
- Sharpness-Aware Data Generation for Zero-shot Quantization (ICML 2024 · similarity 0.91)
- SynQ: Accurate Zero-shot Quantization by Synthesis-aware Fine-tuning (ICLR 2025 · similarity 0.89)
- Zero-Shot Quantization via Weight-Space Arithmetic (NeurIPS 2026 · similarity 0.89)
- Quantization without Tears (CVPR 2025 · similarity 0.87)
- One-Step Forward and Backtrack: Overcoming Zig-Zagging in Loss-Aware Quantization Training (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.