AI Paper Atlas

Research map / Architectures and efficiency / RNNs and normalization

Channel selection using Gumbel Softmax

Charles Herrmann, Richard Strong Bowen, Ramin Zabih

ECCV 2020 · Architectures and efficiency › RNNs and normalization · approach: Differentiable optimization layers · 69 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use filter granularity instead of layer granularity for data-dependent gates (ablation)Gumbel softmax gating on ResNet-50 · ImageNetFLOPs at similar accuracy: 2.72 -> 1.94Improved
use filter granularity instead of layers for pruning (ablation)data-independent gates · ImageNetFLOPs at similar accuracy: 28% improvement in FLOPsImproved
add the batch activation loss instead of AIG per-gate loss (ablation)data-dependent gating · ImageNetFLOPs: approximately 12% boostImproved
prune ResNet-50 with gates at target rate 0.5Ours ind · ImageNettop-1 accuracy: 76.13 -> 76.20Improved
conditional computation at t=.5 on ResNet-50Ours dep · ImageNettop-1 accuracy: 76.13 -> 76.3Improved
conditional computation at t=.4 on ResNet-50Ours dep · ImageNettop-1 accuracy: 76.13 -> 76.04No clear effect
replace AutoPruner with ours at similar FLOPsResNet-50 pruning · ImageNettop-1 accuracy: 74.76 -> 76.14Improved
replace AIG with ours dep on ResNet-110conditional computation · CIFAR-10top-1 accuracy: 94.24 -> 94.36Improved
replace AMC with ours at 50% FLOPsResNet-56 pruning · CIFAR-10accuracy drop: 1.0% -> 0.4%Improved
use straight-through hard gates instead of soft Gumbel softmax gates (ablation)Gumbel gating · trainingperformanceImproved
add early-exit gates to DenseNetDenseNet · CIFAR-10accuracy on selected images: 81.36 -> 96.37Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy