AI Paper Atlas

Research map / Architectures and efficiency / RNNs and normalization

Scaling Up Exact Neural Network Compression by ReLU Stability

Thiago Serra, Xin Yu, Abhinav Kumar, Srikumar Ramalingam

NeurIPS 2021 · Architectures and efficiency › RNNs and normalization · approach: Theory of depth and width · 4 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
solve a single MILP for all neurons (ISA) instead of one per neuronexact compression · MNISTmedian runtime: 83 times fasterImproved
single MILP instead of per-neuron MILPexact compression · CIFAR-10median runtime: 183 times fasterImproved
single MILP instead of per-neuron MILPexact compression · CIFAR-100median runtime: 137 times fasterImproved
increase network size from 2x100 to 2x400ISA vs baseline · MNISTmedian runtime ratio: 77 -> 193Improved
add GPU-based preprocessing step (ablation)ISA · MNIST 2x100median runtime: 3.2 times speed upImproved
add moderate L1 regularizationReLU MLP · MNIST 2x100accuracy: 97.92 -> 98.13Improved
add strong L1 regularization (0.0004)ReLU MLP · MNIST 2x100accuracy: 97.92 -> 97.28Hurt
add L1 regularizationReLU MLP · MNIST 4x100accuracy: 97.93 -> 98.23Improved
increase L1 regularization from 0 to 0.0002LEO++ · MNIST 2x100% neurons removed: 0 -> 18Improved
increase depth from 2x100 to 5x100 at l1=0ISA + LEO++ · MNISTcompression runtime (s): 3.4 -> 2000Hurt
use preprocessing alone to decide stability (inexact)ISA · MNIST% extra neurons removed: 1.16No clear effect
constrain sum of MNIST inputs to [15, 320] instead of [0, 784] (ablation)ISA · MNISTruntime: 17.7% lower geometric meanImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy