Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning
Sharpness-Aware Data Generation for Zero-shot Quantization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SADAG, a zero-shot quantization data generator that uses sharpness-aware gradient matching (approximated via neighbor gradient dissimilarity) plus BN and diversity losses to produce calibration data for low-bit quantization.
Problem
Zero-shot quantization needs synthetic calibration data, but prior generators ignore how the generated data affects the sharpness (generalization) of the quantized model; a real validation set is unavailable.
Approach
Shows that minimizing quantized-model sharpness on validation data can be approximated by maximizing gradient matching between synthetic and real validation reconstruction-loss gradients (last FC layer), and replaces the unavailable validation set with a SAM-like perturbation that finds each sample's least gradient-similar neighbor, combined with BN and diversity losses.
Key results
- On CIFAR-100 ResNet-20, SADAG reaches 65.94% at 3/3 (vs. Genie 65.25%) and 69.11% at 4/4 (vs. Genie 68.35%).
- On ImageNet 2/2, SADAG improves over Genie by 0.77% (ResNet-18, 54.51%), 0.74% (ResNet-50), and 1.08% (MobileNetV2, 13.01%).
- Ablation on generated images (ResNet-18, 2/2): 54.51% with 1,024 images vs. Genie 53.74%; method is about 1.5x slower than Genie.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SADAG instead of Genie | ResNet-20 quantization · CIFAR-100, 3/3 | Top-1 accuracy (%): 65.25 -> 65.94; improves over Genie 0.69% | Helped |
| Use SADAG instead of Genie | ResNet-20 quantization · CIFAR-100, 4/4 | Top-1 accuracy (%): 68.35 -> 69.11; improves over Genie 0.76% | Helped |
| Use SADAG instead of Genie | ImageNet ResNet-18 quantization · ImageNet, 2/2 | Top-1 accuracy (%): 53.74 -> 54.51; improvements are 0.77% | Helped |
| Use SADAG instead of Genie | ImageNet ResNet-50 quantization · ImageNet, 2/2 | Top-1 accuracy (%): 56.81 -> 57.55; improvements are 0.74% | Helped |
| Use SADAG instead of Genie | ImageNet MobileNetV2 quantization · ImageNet, 2/2 | Top-1 accuracy (%): 11.93 -> 13.01; improvements are 1.08% | Helped |
| Use SADAG instead of Genie | ImageNet quantization · ImageNet, 2/4; 3/3; 4/4 across ResNet-18, ResNet-50, and MobileNetV2 | Top-1 accuracy (%) | Helped |
| Increase generated images from 128 to 256 (ablation) | SADAG · ImageNet, ResNet-18, 2/2 | Top-1 accuracy (%): 47.54 -> 51.25; 3.71% | Helped |
| Increase generated images from 256 to 512 (ablation) | SADAG · ImageNet, ResNet-18, 2/2 | Top-1 accuracy (%): 51.25 -> 53.19; 1.94% | Helped |
| Increase generated images from 512 to 1024 (ablation) | SADAG · ImageNet, ResNet-18, 2/2 | Top-1 accuracy (%): 53.19 -> 54.51 | Helped |
| Set lambda1 to 0 instead of 1 (ablation) | SADAG · ImageNet, ResNet-18, 2/2 | Top-1 accuracy (%): 54.51 -> 54.14 | Hurt |
| Set lambda1 to 0.5 instead of 1 (ablation) | SADAG · ImageNet, ResNet-18, 2/2 | Top-1 accuracy (%): 54.51 -> 54.15 | Hurt |
| Set lambda1 to 2 instead of 1 (ablation) | SADAG · ImageNet, ResNet-18, 2/2 | Top-1 accuracy (%): 54.51 -> 53.99 | Hurt |
Limitations
- Requires relaxing the Hessian matrix (assumed to be a scaled identity), an approximation.
- Gradient matching uses only the final fully-connected layer rather than the whole model.
- Gains over Genie are modest (often under 1%).
Experimental setup
Datasets: CIFAR-100 ImageNet
Benchmarks: CIFAR-100 ImageNet
Metrics: Top-1 accuracy
Baselines: Qimera AdaSG IntraQ AdaDFQ Genie
Builds on: Genie SAM AdaRound BRECQ ZeroQ
Closest papers by meaning
- SynQ: Accurate Zero-shot Quantization by Synthesis-aware Fine-tuning (ICLR 2025 · similarity 0.89)
- Zero-Shot Quantization via Weight-Space Arithmetic (NeurIPS 2026 · similarity 0.88)
- Learning from Loss Landscape: Generalizable Mixed-Precision Quantization via Adaptive Sharpness-Aware Gradient Aligning (ICML 2025 · similarity 0.87)
- Enhancing Diversity for Data-free Quantization (CVPR 2025 · similarity 0.86)
- A Generalization Theory for Zero-Shot Prediction (ICML 2025 · similarity 0.86)
- On the Zero-shot Adversarial Robustness of Vision-Language Models: A Truly Zero-shot and Training-free Approach (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.