Research map / Architectures and efficiency / Pruning and efficient tuning
The Knowledge Within: Methods for Data-Free Model Compression
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add BN-statistics matching to synthetic sample generation rather than using Inception-only or Gaussian samples | ResNet-44 quantized distillation · CIFAR10, 2w4a, 50 samples/class | accuracy: 79.62 -> 88.52; 70.03 -> 88.52 | Helped |
| Add BN-statistics matching to synthetic sample generation rather than using Inception-only or Gaussian samples | Wide ResNet-28-10 quantized distillation · CIFAR100, 4w4a, 50 samples/class | accuracy: 53.64 -> 79.21; 64.8 -> 79.21 | Helped |
| Add BN-statistics matching to synthetic sample generation rather than using Inception-only or Gaussian samples | ResNet-18 quantized distillation · ImageNet, 4w4a, 10 samples/class | accuracy: 62.8 -> 67.98; 63.98 -> 67.98 | Helped |
| Combine BN-statistics matching with Inception loss instead of BN-statistics matching alone | ResNet-44 calibration · CIFAR10, 4w4a, 50 samples/class | accuracy: 88.5 -> 89.1 | Helped |
| Combine BN-statistics matching with Inception loss instead of BN-statistics matching alone | ResNet-18 calibration · ImageNet, 4w4a, 10 samples/class | accuracy: 55.29 -> 55.49 | Helped |
| Add intermediate quantization loss to KD (ablation) | ResNet-44 quantized distillation · CIFAR10, 2w4a, 1 sample | accuracy: 76.32 -> 79.67 | Helped |
| Add input mixing to KD (ablation) | ResNet-44 quantized distillation · CIFAR10, 2w4a, 1 sample | accuracy: 76.32 -> 82.49 | Helped |
| Add intermediate quantization loss and input mixing to KD (ablation) | ResNet-44 quantized distillation · CIFAR10, 2w4a, 1 sample | accuracy: 76.32 -> 83.68 | Helped |
| Vary the Gaussian smoothing-kernel sigma in Inception sample generation (ablation) | Inception calibration samples for ResNet-44 · CIFAR10, 4w4a | accuracy: 89.22 (0.22) at 0.375; 87.44 (0.13) at 1.0 | Mixed |
| Vary the Gaussian smoothing-kernel sigma in combined BNS+Inception sample generation (ablation) | BNS+Inception calibration samples for ResNet-44 · CIFAR10, 4w4a | accuracy: 88.87 (0.22) at 0.375; 89.1 (0.09) at 1.0 | Mixed |
Closest papers by meaning
- Data-Free Quantization via Pseudo-label Filtering (CVPR 2024 · similarity 0.88)
- Rate Distortion For Model Compression:From Theory To Practice (ICML 2019 · similarity 0.88)
- Enhancing Diversity for Data-free Quantization (CVPR 2025 · similarity 0.87)
- Forget the Data and Fine-Tuning! Just Fold the Network to Compress (ICLR 2025 · similarity 0.86)
- Effective Interplay between Sparsity and Quantization: From Theory to Practice (ICLR 2025 · similarity 0.84)
- Retraining-Free Model Quantization via One-Shot Weight-Coupling Learning (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.