Research map / Probabilistic and scientific modeling / Bayesian optimization
A Bayesian Optimization Framework for Neural Network Compression
Xingchen Ma, Amal Rannen Triki, Maxim Berman, Christos Sagonas, Jacques Cali, Matthew B. Blaschko
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use the PAC-style acquisition function with Gaussian-noise uncertainty instead of random search | BO compression of ResNet18 · ResNet18, gamma=1.0 | top-1 accuracy (%): 58.2 -> 62.9 | Improved |
| use the proposed BO instead of Fabolas for parameter selection | BO compression of ResNet18 · ResNet18, gamma=1.0 | time to converge (s): 46876 -> 1378 | Improved |
| use the proposed BO instead of Fabolas | BO compression of ResNet18 · ResNet18, gamma=1.0 | top-1 accuracy (%): 61.8 -> 62.9 | Improved |
| use knowledge distillation (function norm) fidelity instead of top-1 error as objective (ablation) | FC3 SVD compression · MNIST FC3, time | time (s): 365.53 -> 79.54 | Improved |
| use knowledge distillation fidelity instead of risk on labeled data (ablation) | FC3 SVD compression · MNIST FC3 | top-1 accuracy (%): 97.73 -> 97.78 | No clear effect |
| fine-tune the tensor-decomposed VGG-16 for 5 epochs | VGG-16 compression · ILSVRC2012 val | top-1 accuracy (%): 59.86% to 69.11% | Improved |
| fine-tune the tensor-decomposed VGG-16 for 5 epochs | VGG-16 compression · ILSVRC2012 val | top-5 accuracy (%): 83.01% to 88.69% | Improved |
| fine-tune after the second (SVD) compression stage | VGG-16 compression · ILSVRC2012 val | top-1 accuracy (%): 65.41% to 68.69% | Improved |
| increase gamma from 0.00006 to 0.0005 (ablation) | ResNet18 low-rank compression · ILSVRC2012 val, no fine-tuning | top-1 accuracy (%): 36.81 -> 68.16 | Improved |
| increase gamma from 0.00006 to 0.0005 (ablation) | ResNet50 low-rank compression · ILSVRC2012 val, no fine-tuning | top-1 accuracy (%): 48.93 -> 74.83 | Improved |
| increase gamma from 0.00006 to 0.0005 (ablation) | ResNet50 low-rank compression · ILSVRC2012 val | compression ratio (%): 17.96 -> 39.55 | Mixed |
| increase gamma beyond 0.0005 (ablation) | ResNet18/ResNet50 compression · ILSVRC2012 val | top-1 accuracy (%) | No clear effect |
Closest papers by meaning
- Bayesian Compression for Deep Learning (NeurIPS 2017 · similarity 0.91)
- A Study of Bayesian Neural Network Surrogates for Bayesian Optimization (ICLR 2024 · similarity 0.88)
- BaSIC: BayesNet Structure Learning for Computational Scalable Neural Image Compression (ECCV 2024 · similarity 0.88)
- Does Compression Imply Generalization? A Minimum Description Length Perspective (NeurIPS 2026 · similarity 0.87)
- Generalization Bounds via Meta-Learned Model Representations: PAC-Bayes and Sample Compression Hypernetworks (ICML 2025 · similarity 0.87)
- Implicit Compressibility of Overparametrized Neural Networks Trained with Heavy-Tailed SGD (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
