Research map / Architectures and efficiency / Pruning and efficient tuning
CrAM: A Compression-Aware Minimizer
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train with CrAM⁺-Multi rather than SGD | ResNet50 · ImageNet val, dense | accuracy: 76.95% -> 77.3% | Helped |
| Train with CrAM⁺-Multi and prune one-shot at 90% sparsity | ResNet50 · ImageNet val, 90% sparsity | accuracy: 74.7 vs. 75.3 | Mixed |
| Finetune with CrAM⁺-Multi for 20 epochs instead of 10 | ResNet18 · ImageNet val, dense | accuracy: 70.4 -> 70.6 | Helped |
| Finetune with CrAM⁺-Multi instead of Adam | BERT-base · SQuADv1.1, 50% sparsity | F1: 80.0 -> 88.3 | Helped |
| One-shot prune CrAM⁺-Multi with oBERT and finetune the sparse model | BERT-base · SQuADv1.1, 80% sparsity | F1: 84.9 -> 87.4 | Helped |
| Use CrAM⁺-k95 rather than SFW | VGG-16 · CIFAR-10 test, 95% sparsity | accuracy: 92.0 -> 94.1 ± 0.1 | Helped |
| Remove the CrAM gradient-ascent perturbation step (ablation) | CrAM⁺-Multi · CIFAR-10/ResNet20, 90% sparsity | accuracy: 90.1 ± 0.2 -> 90.0 ± 0.2 | No clear effect |
| Remove the CrAM gradient-ascent perturbation step (ablation) | CrAM⁺-k70 · CIFAR-10/ResNet20, 70% sparsity | accuracy: 92.7 ± 0.0 -> 92.4 ± 0.1 | Hurt |
| Replace CrAM with C-SAM (ablation) | ResNet20 · CIFAR-10, dense model and one-shot pruning | accuracy: CrAM-k70 dense 92.4 ± 0.2; C-SAM-k70 dense 91.4 ± 0.0; at 90% sparse 81.1 ± 1.3 -> 85.3 ± 0.5 | Mixed |
| Use sparse intermediate gradients (ablation) | CrAM / CrAM⁺ · ImageNet one-shot pruning | accuracy | Helped |
| Use sparse gradients during CrAM-k50 training (ablation) | CrAM-k50 · CIFAR-10, dense model | accuracy: 93.4% -> 92.8% | Hurt |
| Add explicit dense-model loss optimization to CrAM (ablation) | CrAM⁺ · ImageNet, high sparsity training | dense-model accuracy | Helped |
Closest papers by meaning
- Auto-Compressing Networks (NeurIPS 2025 · similarity 0.86)
- Language Models as Zero-shot Lossless Gradient Compressors: Towards General Neural Parameter Prior Models (NeurIPS 2024 · similarity 0.86)
- SqueezeLLM: Dense-and-Sparse Quantization (ICML 2024 · similarity 0.86)
- EPSD: Early Pruning with Self-Distillation for Efficient Model Compression (AAAI 2024 · similarity 0.86)
- SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression (ICLR 2024 · similarity 0.86)
- CRONOS: Enhancing Deep Learning with Scalable GPU Accelerated Convex Neural Networks (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.