Research map / Architectures and efficiency / CNN design
SplitNet: Learning to Semantically Split Deep Networks for Parameter Reduction and Model Parallelization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Learned semantic splitting (SplitNet) vs. random class splits (ablation) | WRN-16-8 deep split on CIFAR-100 · CIFAR-100 | test error (%): 27.12 -> 24.62 | Helped |
| Shallow split of last 5 conv layers in WRN-16-8 | WRN-16-8 · CIFAR-100 | parameter reduction (%): 32.54% fewer parameters with error 23.96 vs 24.28 baseline | Helped |
| Deep split (layers from 11) in WRN-16-8 with dropout | WRN-16-8 · CIFAR-100 | test error (%): 24.62 vs 24.52 dropout baseline; 46.39% params reduced | Mixed |
| FC-only split of output layer (ablation) | WRN-16-8 · CIFAR-100 | parameter reduction (%): 0.35% params reduced; error 24.26 vs 24.28 | No clear effect |
| Splitting AlexNet fc layers (2-4-8) with learned groups | AlexNet · ILSVRC2012 | test error (%): 41.72 -> 43.02 (+1.30) with 56.17% params reduced | Mixed |
| Random class splits instead of learned splits on AlexNet (ablation) | AlexNet 2-4-8 · ILSVRC2012 | test error (%): 43.02 -> 43.35 | Hurt |
| Two-GPU model parallel with redundant lower layers (deep split) | WRN-16-8 test-time · CIFAR-100 evaluation time | speedup (x): 1.73x speedup | Helped |
| Assign lower layers to separate GPU (3-way deep split) (ablation) | WRN-16-8 test-time · CIFAR-100 evaluation time | speedup (x): 2.22x vs 1.73x with redundant lower layers | Helped |
| Horizontal model split without pruning inter-group connections (ablation) | WRN-16-8 test-time · CIFAR-100 evaluation time | speedup (x): 0.52x | Hurt |
Closest papers by meaning
- Mesh-TensorFlow: Deep Learning for Supercomputers (NeurIPS 2018 · similarity 0.88)
- Efficient Distributed Inference of Deep Neural Networks via Restructuring and Pruning (AAAI 2023 · similarity 0.87)
- Exploring Hidden Dimensions in Accelerating Convolutional Neural Networks (ICML 2018 · similarity 0.87)
- Refining Architectures of Deep Convolutional Neural Networks (CVPR 2016 · similarity 0.87)
- LayerMerge: Neural Network Depth Compression through Layer Pruning and Merging (ICML 2024 · similarity 0.86)
- GDeR: Safeguarding Efficiency, Balancing, and Robustness via Prototypical Graph Pruning (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.