Research map / Label-efficient and robust learning / Recognition and long-tail
Data-Free Quantization via Pseudo-label Filtering
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a data-free quantization method that scores synthetic images by self-entropy of the pre-trained network, splits them into high- and low-reliability sets, and trains the quantized ResNet/MobileNet with major and auxiliary pseudo-labels plus knowledge distillation.
Problem
Data-free quantization relies on synthetic data that differs from the original training data, and existing methods use all synthetic samples without assessing their reliability, which can mislead quantized network training.
Approach
Use the pre-trained network's self-entropy on each synthetic sample as a reliability metric, filter samples with a decaying threshold, label reliable samples with their top prediction (major pseudo-label) and unreliable ones with top-2 soft pseudo-labels (auxiliary), and train the quantized model with these plus KL and feature MSE losses.
Key results
- Achieves 92.47% (W4A4) and 88.04% (W3A3) on CIFAR-10 ResNet-20, and 66.94% / 57.03% on CIFAR-100, best in most settings.
- On ImageNet ResNet-18 W4A4 reaches 67.02%, above HAST (66.91%) and IntraQ (66.47%).
- Adding the proposed pseudo-label loss to GDFQ and IntraQ improves them by 0.43 to 0.76 points on CIFAR-100 and ImageNet.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use pseudo-label filtering and training | data-free quantization · CIFAR-10, ResNet-20 W4A4 | Top-1 accuracy: 92.36 -> 92.47 | Helped |
| use pseudo-label filtering and training | data-free quantization · CIFAR-10, ResNet-20 W3A3 | Top-1 accuracy: 88.34 -> 88.04 | Hurt |
| use pseudo-label filtering and training | data-free quantization · CIFAR-100, ResNet-20 W4A4 and W3A3 | Top-1 accuracy: 66.81 -> 66.94; 55.67 -> 57.03 | Helped |
| use pseudo-label filtering and training | data-free quantization · ImageNet, ResNet-18 W5A5 and W4A4 | Top-1 accuracy: 70.29 -> 70.35; 66.91 -> 67.02 | Helped |
| use pseudo-label filtering and training | data-free quantization · ImageNet, MobileNet-V1 W5A5 and W4A4 | Top-1 accuracy: 68.52 -> 69.44; 57.70 -> 59.51 | Helped |
| use pseudo-label filtering and training | data-free quantization · ImageNet, ResNet-50 W5A5 and W4A4 | Top-1 accuracy: 76.03 -> 76.08; 68.58 -> 68.97 | Helped |
| add high- and low-reliable pseudo-label losses (ablation) | quantized ResNet-20 · CIFAR-100 W4A4 | Top-1 accuracy: 66.32 -> 66.94 | Helped |
| add low-reliable pseudo-label loss (ablation) | quantized ResNet-20 with high-reliable pseudo-label loss · CIFAR-100 W4A4 | Top-1 accuracy: 66.68 -> 66.94 | Helped |
| add high-reliable pseudo-label loss (ablation) | quantized ResNet-20 with low-reliable pseudo-label loss · CIFAR-100 W4A4 | Top-1 accuracy: 66.57 -> 66.94 | Helped |
| add feature MSE loss (ablation) | quantized ResNet-20 with both pseudo-label losses · CIFAR-100 W4A4 | Top-1 accuracy: 66.52 -> 66.94 | Helped |
| add feature MSE loss (ablation) | quantized ResNet-20 with high-reliable pseudo-label loss · CIFAR-100 W4A4 | Top-1 accuracy: 66.30 -> 66.68 | Helped |
| add feature MSE loss (ablation) | quantized ResNet-20 with no pseudo-label losses · CIFAR-100 W4A4 | Top-1 accuracy: 65.81 -> 66.32 | Helped |
Surprising findings
- Synthetic data has consistently higher self-entropy than original data for the pre-trained network, so most synthetic samples are unreliable.
- Using only low-reliability samples with auxiliary soft labels (N6) gives lower accuracy than using high-reliability samples alone (N5), as in the ablation.
Limitations
- Reliability threshold and loss weights (Tl, Tu, beta, mu, tau, gamma) are hand-tuned per dataset.
- Gains over strongest baselines are often small (e.g. +0.11% over HAST on CIFAR-10 4-bit), and the method is not better on 3-bit CIFAR-10.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 ImageNet (ILSVRC2012)
Benchmarks: CIFAR-10 CIFAR-100 ImageNet
Metrics: Top-1 accuracy
Baselines: ZeroQ GDFQ DSG Qimera ARC ARC+AIT IntraQ AdaSG AdaDFQ HAST ZAQ SQuant
Builds on: ZeroQ IntraQ GDFQ knowledge distillation
Closest papers by meaning
- The Knowledge Within: Methods for Data-Free Model Compression (CVPR 2020 · similarity 0.88)
- Robustness-Guided Image Synthesis for Data-Free Quantization (AAAI 2024 · similarity 0.86)
- Bridging the Gap between Gaussian Diffusion Models and Universal Quantization for Image Compression (CVPR 2025 · similarity 0.84)
- Enhancing Diversity for Data-free Quantization (CVPR 2025 · similarity 0.84)
- SynQ: Accurate Zero-shot Quantization by Synthesis-aware Fine-tuning (ICLR 2025 · similarity 0.84)
- Quantization without Tears (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.