Research map / Architectures and efficiency / RNNs and normalization
Channel selection using Gumbel Softmax
Charles Herrmann, Richard Strong Bowen, Ramin Zabih
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use filter granularity instead of layer granularity for data-dependent gates (ablation) | Gumbel softmax gating on ResNet-50 · ImageNet | FLOPs at similar accuracy: 2.72 -> 1.94 | Improved |
| use filter granularity instead of layers for pruning (ablation) | data-independent gates · ImageNet | FLOPs at similar accuracy: 28% improvement in FLOPs | Improved |
| add the batch activation loss instead of AIG per-gate loss (ablation) | data-dependent gating · ImageNet | FLOPs: approximately 12% boost | Improved |
| prune ResNet-50 with gates at target rate 0.5 | Ours ind · ImageNet | top-1 accuracy: 76.13 -> 76.20 | Improved |
| conditional computation at t=.5 on ResNet-50 | Ours dep · ImageNet | top-1 accuracy: 76.13 -> 76.3 | Improved |
| conditional computation at t=.4 on ResNet-50 | Ours dep · ImageNet | top-1 accuracy: 76.13 -> 76.04 | No clear effect |
| replace AutoPruner with ours at similar FLOPs | ResNet-50 pruning · ImageNet | top-1 accuracy: 74.76 -> 76.14 | Improved |
| replace AIG with ours dep on ResNet-110 | conditional computation · CIFAR-10 | top-1 accuracy: 94.24 -> 94.36 | Improved |
| replace AMC with ours at 50% FLOPs | ResNet-56 pruning · CIFAR-10 | accuracy drop: 1.0% -> 0.4% | Improved |
| use straight-through hard gates instead of soft Gumbel softmax gates (ablation) | Gumbel gating · training | performance | Improved |
| add early-exit gates to DenseNet | DenseNet · CIFAR-10 | accuracy on selected images: 81.36 -> 96.37 | Improved |
Closest papers by meaning
- Importance Estimation for Neural Network Pruning (CVPR 2019 · similarity 0.87)
- Fire Together Wire Together: A Dynamic Pruning Approach With Self-Supervised Mask Prediction (CVPR 2022 · similarity 0.87)
- Pruning from Scratch (AAAI 2020 · similarity 0.87)
- Network Pruning via Performance Maximization (CVPR 2021 · similarity 0.87)
- A Main/Subsidiary Network Framework for Simplifying Binary Neural Networks (CVPR 2019 · similarity 0.87)
- DFPC: Data flow driven pruning of coupled channels without data. (ICLR 2023 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
