Research map / Architectures and efficiency / Pruning and efficient tuning
Rate Distortion For Model Compression:From Theory To Practice
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the weight-importance objective instead of squared weight error for unsupervised pruning (ablation) | Pruning · MNIST, CIFAR10, CIFAR100 | test accuracy | Mixed |
| Use the weight-importance objective instead of squared weight error for unsupervised pruning (ablation) | Pruning · MNIST, CIFAR10, CIFAR100 | cross-entropy loss | Helped |
| Use weighted k-means with the weight-importance objective instead of squared weight error for unsupervised quantization (ablation) | Quantization · MNIST, CIFAR10, CIFAR100 | test accuracy | Mixed |
| Use weighted k-means with the weight-importance objective instead of squared weight error for unsupervised quantization (ablation) | Quantization · MNIST, CIFAR10, CIFAR100 | cross-entropy loss | Helped |
| Use the gradient-based objective instead of squared weight error for supervised pruning (ablation) | Pruning · MNIST, CIFAR10 | test accuracy | Helped |
| Use the gradient-based objective instead of squared weight error for supervised quantization (ablation) | Quantization · MNIST, CIFAR10 | test accuracy | Helped |
| Use the Hessian-based objective instead of squared weight error for supervised pruning (ablation) | Pruning · MNIST, CIFAR10 | test accuracy | Helped |
| Use the Hessian-based objective instead of squared weight error for supervised quantization (ablation) | Quantization · MNIST, CIFAR10 | test accuracy | Helped |
| Add the Hessian-based quartic term to the gradient-based objective (ablation) | Supervised pruning and quantization · MNIST, CIFAR10 | test accuracy | Helped |
| Use the gradient-based objective instead of squared weight error for supervised pruning (ablation) | Pruning · CIFAR100 | accuracy | Helped |
| Use the gradient-based objective instead of squared weight error for supervised quantization (ablation) | Quantization · CIFAR100 | accuracy | Helped |
| Increase temperature from 1, initially (ablation) | Supervised pruning · MNIST | accuracy | Mixed |
Closest papers by meaning
- Compressing Neural Networks: Towards Determining the Optimal Layer-wise Decomposition (NeurIPS 2021 · similarity 0.90)
- Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruning (NeurIPS 2022 · similarity 0.90)
- Forget the Data and Fine-Tuning! Just Fold the Network to Compress (ICLR 2025 · similarity 0.88)
- Effective Interplay between Sparsity and Quantization: From Theory to Practice (ICLR 2025 · similarity 0.88)
- Optimal Neural Compressors for the Rate-Distortion-Perception Tradeoff (NeurIPS 2025 · similarity 0.88)
- Approaching Rate-Distortion Limits in Neural Compression with Lattice Transform Coding (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.