AI Paper Atlas

Research map / Architectures and efficiency / RNNs and normalization

Shakeout: A New Approach to Regularized Deep Neural Network Training

Guoliang Kang, Jun Li, Dacheng Tao

TPAMI 2017 · Architectures and efficiency › RNNs and normalization · approach: Representation geometry and collapse · 64 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use Shakeout instead of Dropout when data is scarceFC network · MNIST (train size 500)test error (%): 11.76 -> 10.81Improved
use Shakeout instead of Dropout when data is scarceCNN · MNIST (train size 500)test error (%): 6.16 -> 4.83Improved
use Shakeout instead of Dropout with plentiful dataFC network · MNIST (train size 8000)test error (%): 2.95 -> 2.96No clear effect
use Shakeout instead of Dropout with plentiful dataCNN · MNIST (train size 8000)test error (%): 1.23 -> 1.31Hurt
use Shakeout instead of DropoutAlexFastNet · CIFAR-10 (train size 300)test error (%): 65.34 -> 63.71Improved
use Shakeout instead of DropoutAlexFastNet · CIFAR-10 (train size 5500)test error (%): 36.01 -> 34.54Improved
use Shakeout instead of Dropout with full dataAlexFastNet · CIFAR-10 (train size 40000)test error (%): 20.50 -> 20.56No clear effect
use Shakeout instead of DropoutWRN-16-4 · CIFAR-10 (train size 40000)test error (%): 9.32 -> 9.01Improved
add Dropout instead of standard BPFC network · MNIST (train size 500)test error (%): 13.66 -> 11.76Improved
use Shakeout instead of standard BPWRN-16-4 · CIFAR-10 (train size 50000)test error (%): 14.39 -> 7.97Improved
use Shakeout instead of Dropout on AlexNet FC7 and FC8AlexNet · ImageNet-2012 valtop-1 error (%): 42.99 -> 42.88No clear effect
use Shakeout instead of Dropout for weight pruning at ratio 96%AlexNet · ImageNet-2012 valrelative accuracy loss: 22.50%Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy