AI Paper Atlas

Research map / Architectures and efficiency / CNN design

Learning Compressed Transforms with Low Displacement Rank

Anna Thomas, Albert Gu, Tri Dao, Atri Rudra, Christopher Ré

NeurIPS 2018 · Architectures and efficiency › CNN design · approach: Invertible and wide-network analysis · 3 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Learn displacement operators rather than fixing them to the Toeplitz-like operatorsLDR-SD weight layers · MNIST-bg-rot, MNIST-noise, CIFAR-10, and NORBtest accuracy: 3.14, 2.70, 3.55, and 3.37 accuracy pointsImproved
Use learned operators instead of fixed operators at rank oneSingle hidden layer network · MNIST-bg-rot, MNIST-noise, and NORBtest accuracy: 1.73, 13.30, and 2.92 accuracy pointsImproved
Increase LDR-SD displacement rank to 16 (ablation)LDR-SD fully-connected layers · MNIST-bg-rot, MNIST-noise, CIFAR-10, and NORBtest accuracy: 5.87, 15.05, 0.74, and 6.86 accuracy pointsImproved
Learn subdiagonal operators instead of using fixed Toeplitz-like operatorsLSTM input-hidden weights · WikiText-2test perplexity: 19.94-42.92 perplexity pointsImproved
Replace convolutional channels and pooling with two generic LDR-SD matricesCIFAR-10 CNN · CIFAR-10test accuracy: within 1.05 accuracy pointsHurt
Use learned LDR-SD channels rather than fixed Toeplitz-like channels at rank 16Two structured-matrix layers · CIFAR-10test accuracy: 57.29 -> 59.36Improved
Increase the Toeplitz-like rank from 1 to 2 rather than broaden the operator class to LDR-SD at rank 1 (ablation)Compressed fully-connected layer · Image classification datasetstest accuracyHurt
Train the structured LDR-TD model with less training dataSingle hidden layer network · MNIST-noisetest accuracyImproved
Train the structured LDR-TD model with less training dataCNN · MNIST-noisetest accuracyImproved
Replace the LSTM input-hidden matrix with LDR-SD rather than an unstructured layerSingle-layer LSTM · WikiText-2test perplexityHurt
Use LDR-SD instead of an unstructured fully-connected layer at rank 1Single hidden layer network · CIFAR-10test accuracyHurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy