Research map / Architectures and efficiency / Pruning and efficient tuning
Compact Neural Network via Stacking Hybrid Units
Weichao Lan, Yiu‐ming Cheung, Juyong Jiang, Zhikai Hu, Mengke Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase unit hidden width c_h from 4 to 6 (ablation) | TissueNet VGG-16 · CIFAR-100 | accuracy (%): 70.40 -> 71.13 | Improved |
| keep first and last blocks as normal layers (r strategy) (ablation) | TissueNet ResNet-18 · CIFAR-100 | accuracy (%): 75.07 -> 76.56 | Improved |
| keep first and last blocks as normal layers (r strategy) (ablation) | TissueNet ResNet-18 · CIFAR-10 | accuracy (%): 93.31 -> 94.50 | Improved |
| replace only intermediate layers (r strategy) (ablation) | TissueNet VGG-19 · Tiny-ImageNet | top-1 accuracy (%): 50.20 -> 53.54 | Improved |
| replace only intermediate layers (r strategy) (ablation) | TissueNet ResNet-18 · CIFAR-100 | Computation Efficiency (CE): 2.39 -> 1.22 | Hurt |
| raise remaining FLOPs from 21% to 33% and params from 10% to 30% (ablation) | TissueNet ResNet-style · CIFAR-100 | accuracy (%): 72.87% -> 74.51% | Improved |
| use hybrid units (Hybrid2) instead of single unit U2 at equal FLOPs (ablation) | TissueNet LeNet-5-style · MNIST | accuracy (%): 98.31 -> 98.65 | Improved |
| use hybrid units (Hybrid1) instead of single unit U1 (ablation) | TissueNet LeNet-5-style · MNIST | accuracy (%): 98.61 -> 98.47 | No clear effect |
| use hybrid units (Hybrid3) instead of single unit U3 (ablation) | TissueNet LeNet-5-style · MNIST | accuracy (%): 98.72 -> 98.59 | No clear effect |
| TissueNet vs pruning baselines on CE | TissueNet VGG-16 · CIFAR-100 | Computation Efficiency (CE): 1.59 -> 4.74 | Improved |
| TissueNet vs pruning baselines on CE | TissueNet MLP · MNIST | Computation Efficiency (CE): 6.42 -> 36.06 | Improved |
| TissueNet vs CHIP on storage efficiency | TissueNet VGG-16 · CIFAR-10 | Storage Efficiency (SE): 7.82 -> 3.67 | Hurt |
Closest papers by meaning
- “Learning-Compression” Algorithms for Neural Net Pruning (CVPR 2018 · similarity 0.87)
- Data-Independent Neural Pruning via Coresets (ICLR 2020 · similarity 0.86)
- DARB: A Density-Adaptive Regular-Block Pruning for Deep Neural Networks (AAAI 2020 · similarity 0.86)
- Pruning Neural Networks via Coresets and Convex Geometry: Towards No Assumptions (NeurIPS 2022 · similarity 0.86)
- Pruning from Scratch (AAAI 2020 · similarity 0.85)
- Towards Meta-Pruning via Optimal Transport (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
