AI Paper Atlas

Research map / Architectures and efficiency / RNNs and normalization

Swapout: Learning an ensemble of deep architectures

Saurabh Singh, Derek Hoiem, David Forsyth

NeurIPS 2016 · Architectures and efficiency › RNNs and normalization · approach: Representation geometry and collapse · 121 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Replace ResNet v1 training with Swapout at the same widthResNet v1, W × 1 · CIFAR-10error (%): 8.54 -> 8.27Improved
Replace ResNet v2 training with Swapout at the same widthResNet v2 Ours, W × 1 · CIFAR-10error (%): 8.27 -> 7.97Improved
Train with Swapout instead of ResNet v2 at W × 2ResNet v2 Ours, W × 2 · CIFAR-10error (%): 6.54 -> 5.68Improved
Use stochastic inference rather than deterministic inference (ablation)Swapout v2 (20) W × 2 · CIFAR-10error (%): 6.40 -> 5.68Improved
Increase width from W × 2 to W × 4 (ablation)Swapout v2 (20) · CIFAR-10error (%): 5.68 -> 5.09Improved
Increase depth from 20 to 32 layers at W × 4 (ablation)Swapout v2 · CIFAR-10error (%): 5.09 -> 4.76Improved
Use Swapout v2 (32) W × 4 instead of ResNet v2 (1001)CIFAR-10 image classification · CIFAR-10error (%): 4.92 -> 4.76Improved
Train the 20-layer W × 2 model with Swapout rather than ResNet v2ResNet v2 Ours (20) W × 2 · CIFAR-100error (%): 28.08 -> 25.86Improved
Use a wider 32-layer Swapout model rather than ResNet v2 (1001)CIFAR-100 image classification · CIFAR-100error (%): 22.71 -> 22.72No clear effect
Set θ1=θ2=0.5 across blocks instead of using Linear(1, 0.5) (ablation)Swapout v2 (20) W × 2 · CIFAR-10stochastic error (%): 5.68 -> 6.69Hurt
Use θ1=0.2 and θ2=0.8 instead of Linear(1, 0.5) (ablation)Swapout v2 (20) W × 2 · CIFAR-10stochastic error (%): 5.68 -> 7.63Hurt
Use θ1=θ2=Linear(0.5, 1) instead of Linear(1, 0.5) (ablation)Swapout v2 (20) W × 2 · CIFAR-10stochastic error (%): 5.68 -> 6.52Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy