AI Paper Atlas

Research map / Architectures and efficiency / RNNs and normalization

Train-by-Reconnect: Decoupling Locations of Weights from Their Values

Yushi Qiu, Reiji Suda

NeurIPS 2020 · Architectures and efficiency › RNNs and normalization · approach: Representation geometry and collapse · 3 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Train by reconnecting randomly initialized weights with LaPermConv7 · MNIST testtest error: ~0.24% test error; slightly outperformed Adam and LAImproved
Use a shared weight value and reconnect the pruned networkF1 · MNISTaccuracy: ~85.5%Improved
Use a shared weight value and reconnect the pruned networkF2 · MNISTaccuracy: ~53%Improved
Prune F2 matrices according to their parameter counts instead of using the same pruning rate (ablation)LaPerm F2 · MNIST testtest accuracy: ~53% -> 78.14%Improved
Use k=1 synchronization instead of a longer synchronization period (ablation)LaPerm · CIFAR-10, Conv4accuracyHurt
Increase the synchronization period (ablation)LaPerm · CIFAR-10, Conv4final accuracyImproved
Synchronize only once at the end of training (k=90000) (ablation)LaPerm · CIFAR-10, Conv4accuracy: 13.8%Hurt
Initialize with shuffled Adam-trained weights instead of random initialization (ablation)LaPerm · MNIST, Conv7validation accuracyHurt
Initialize with He uniform or normal instead of Glorot uniform or normal (ablation)LaPerm · MNIST, Conv7validation accuracy: ~0.1%Improved
Randomly prune 50% of Conv4's initial weights before reconnecting (ablation)LaPerm · CIFAR-10, Conv4accuracyNo clear effect
Freeze zero connections and train the remaining weights with Adam instead of reconnecting (ablation)LaPerm sparse-network setting · CIFAR-10, Conv4accuracyHurt
Randomly prune up to 70% of initial weights and reconnect (ablation)LaPerm · CIFAR-10, tested architecturesaccuracyNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy