Research map / Architectures and efficiency / RNNs and normalization
Minimally Invasive Surgery for Sparse Neural Networks in Contrastive Manner
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add recovered-model distillation to the compressed-model training objective (ablation) | MIS ResNet-50 70% sparse · ImageNet | Top-1 accuracy (%): 75.950 -> 76.558 | Helped |
| Use dense-model predictions instead of ground-truth labels (ablation) | MIS ResNet-50 70% sparse · ImageNet | Top-1 accuracy (%): 76.558 -> 75.910 | Hurt |
| Use a more accurate ResNeXt-101 dense teacher rather than ResNet-50 (ablation) | MIS compressed ResNet-50 · ImageNet | Top-1 accuracy (%): 76.496 -> 77.038 | Helped |
| Apply MIS instead of pruning alone | SRResNet at 50% sparsity · Set5 | PSNR: 31.234 -> 31.484 | Helped |
| Apply MIS rather than AGP compression alone | Faster R-CNN (R50-1x), 50% sparse · COCO | Box AP: 38.58(±0.11) -> 38.76(±0.14) | Helped |
| Apply MIS rather than pruning alone | GNMT at 50% sparsity · WMT16 En-Ge | BLEU Score: 24.77(±0.16) -> 24.73(±0.15) | No clear effect |
| Apply MIS rather than pruning alone | Transformer at 50% sparsity · WMT14 En-Ge | BLEU Score: 28.89(±0.11) -> 28.91(±0.12) | No clear effect |
Closest papers by meaning
- Dynamic Network Surgery for Efficient DNNs (NeurIPS 2016 · similarity 0.85)
- Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruning (NeurIPS 2022 · similarity 0.83)
- The Iterative Optimal Brain Surgeon: Faster Sparse Recovery by Leveraging Second-Order Information (NeurIPS 2024 · similarity 0.82)
- "Lossless" Compression of Deep Neural Networks: A High-dimensional Neural Tangent Kernel Approach (NeurIPS 2022 · similarity 0.82)
- Compressing Neural Networks: Towards Determining the Optimal Layer-wise Decomposition (NeurIPS 2021 · similarity 0.81)
- HALOC: Hardware-Aware Automatic Low-Rank Compression for Compact Neural Networks (AAAI 2023 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.