Research map / Architectures and efficiency / Pruning and efficient tuning
RED : Looking for Redundancies for Data-FreeStructured Compression of Deep Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase hashing contrast τ from 0 to 20 (ablation) | RED · ImageNet (ResNet-50) | parameters removed (%): 39.6 -> 56.9 | Helped |
| Increase hashing contrast τ from 0 to 20 (ablation) | RED · ImageNet (ResNet-50) | Top-1 accuracy (%): 76.1 -> 67.9 | Hurt |
| Add scalar hashing before merging (ablation) | neuron merging · CIFAR-10 (ResNet-20) | parameters removed (%): 18.58 -> 41.03 | Helped |
| Add uneven depthwise separation after hashing and relaxed merging (ablation) | RED · CIFAR-10 (ResNet-56) | parameters removed (%): 77.68 -> 84.52 | Helped |
| Add uneven depthwise separation without hashing (ablation) | merging-based pruning · CIFAR-10 (ResNet-56) | parameters removed (%): 61.19 -> 61.19 | No clear effect |
| Use block rather than linear descending layer-wise α allocation (ablation) | RED · CIFAR-10 (ResNet-56) | parameters removed (%): 77.90 -> 84.52 | Helped |
| Use block rather than constant layer-wise α allocation (ablation) | RED · CIFAR-10 (ResNet-56) | parameters removed (%): 78.69 -> 84.52 | Helped |
| Use block rather than linear ascending layer-wise α allocation (ablation) | RED · CIFAR-10 (ResNet-56) | parameters removed (%): 80.35 -> 84.52 | Helped |
| Apply hashing with τ=0 | ResNet-56 · CIFAR-10 | logit modification error: 0.75 | No clear effect |
| Apply RED rather than the closest data-free pruning contender | ResNet-56 · CIFAR-10 | parameters removed (%): +44% | Helped |
Closest papers by meaning
- Convolutional Neural Network Pruning With Structural Redundancy Reduction (CVPR 2021 · similarity 0.92)
- Accelerate CNNs from Three Dimensions: A Comprehensive Pruning Framework (ICML 2021 · similarity 0.90)
- Structured Pruning of Neural Networks With Budget-Aware Regularization (CVPR 2019 · similarity 0.89)
- Learning to Prune Deep Neural Networks via Layer-wise Optimal Brain Surgeon (NeurIPS 2017 · similarity 0.88)
- DARB: A Density-Adaptive Regular-Block Pruning for Deep Neural Networks (AAAI 2020 · similarity 0.88)
- UPDP: A Unified Progressive Depth Pruner for CNN and Vision Transformer (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.