AI Paper Atlas

Research map / Architectures and efficiency / CNN design

Inference, Learning and Attention Mechanisms that Exploit and Preserve Sparsity in CNNs

Timo Häckel, Mikhail Usvyatsov, Silvano Galliani, Jan Dirk Wegner, Konrad Schindler B

IJCV 2020 · Architectures and efficiency › CNN design · approach: Filter design and trained-filter analysis · 14 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use sparse direct convolution instead of dense TensorFlow convolutionConvolution layer · Random tensors r=256, filter density 0.1speedup: 14x faster with density regularisation; 7x faster including k-selectionImproved
use sparse representation instead of dense at 512^3Convolution layer · Random tensors memorymemory (GB): 137.28 -> 0.8Improved
use sparse network instead of OctNet at 256^3Sparse network · Modelnet40 training memorymemory: OctNet out of memory; sparse 2880 MBImproved
use sparse network instead of OctNet at low resolution 16^3Sparse network · Modelnet40 training timetraining time per epoch: 16s -> 2m10sHurt
use sparse network instead of OctNet at 128^3Sparse network · Modelnet40 training timetraining time per epoch: 34m43s -> 6m20sImproved
loosen density upper bound from 0.035 to 0.075 (ablation)Sparse net · Thresholded MNIST, 10 epochsaccuracy: 0.954 -> 0.976Improved
loosen density upper bound from 0.035 to 0.075 (ablation)Sparse net · Thresholded MNIST, 60 epochsaccuracy: 0.985 -> 0.992Improved
add density regularisation and pruning (lambda=0.3)Sparse net · Modelnet40 runtime after 90 epochsruntime: 51% fasterImproved
add density regularisation and pruning (lambda=0.3)Sparse net · Modelnet40 accuracyaccuracyNo clear effect
select k largest signed responses instead of largest absolute values (ablation)k-selection attention · Modelnet40 16^3accuracyImproved
tighten density upper bound from 1.0 to 0.06 (ablation)Sparse net · Modelnet40 16^3accuracy: drop reaches approximately 3 percent pointsHurt
use sparse network instead of dense at 256^3Sparse network · ScanNet classification training speedspeedup: more than 10x fasterImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy