Research map / Architectures and efficiency / RNNs and normalization
Train-by-Reconnect: Decoupling Locations of Weights from Their Values
Yushi Qiu, Reiji Suda
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train by reconnecting randomly initialized weights with LaPerm | Conv7 · MNIST test | test error: ~0.24% test error; slightly outperformed Adam and LA | Improved |
| Use a shared weight value and reconnect the pruned network | F1 · MNIST | accuracy: ~85.5% | Improved |
| Use a shared weight value and reconnect the pruned network | F2 · MNIST | accuracy: ~53% | Improved |
| Prune F2 matrices according to their parameter counts instead of using the same pruning rate (ablation) | LaPerm F2 · MNIST test | test accuracy: ~53% -> 78.14% | Improved |
| Use k=1 synchronization instead of a longer synchronization period (ablation) | LaPerm · CIFAR-10, Conv4 | accuracy | Hurt |
| Increase the synchronization period (ablation) | LaPerm · CIFAR-10, Conv4 | final accuracy | Improved |
| Synchronize only once at the end of training (k=90000) (ablation) | LaPerm · CIFAR-10, Conv4 | accuracy: 13.8% | Hurt |
| Initialize with shuffled Adam-trained weights instead of random initialization (ablation) | LaPerm · MNIST, Conv7 | validation accuracy | Hurt |
| Initialize with He uniform or normal instead of Glorot uniform or normal (ablation) | LaPerm · MNIST, Conv7 | validation accuracy: ~0.1% | Improved |
| Randomly prune 50% of Conv4's initial weights before reconnecting (ablation) | LaPerm · CIFAR-10, Conv4 | accuracy | No clear effect |
| Freeze zero connections and train the remaining weights with Adam instead of reconnecting (ablation) | LaPerm sparse-network setting · CIFAR-10, Conv4 | accuracy | Hurt |
| Randomly prune up to 70% of initial weights and reconnect (ablation) | LaPerm · CIFAR-10, tested architectures | accuracy | No clear effect |
Closest papers by meaning
- HyperNet Fields: Efficiently Training Hypernetworks without Ground Truth by Learning Weight Trajectories (CVPR 2025 · similarity 0.87)
- RECAST: Reparameterized, Compact weight Adaptation for Sequential Tasks (ICLR 2025 · similarity 0.86)
- Diffusion-based Neural Network Weights Generation (ICLR 2025 · similarity 0.86)
- The Effect of Weight Precision on the Neuron Count in Deep ReLU Networks (ICML 2024 · similarity 0.85)
- Feature Learning Dynamics in Infinite-Depth Neural Networks (NeurIPS 2026 · similarity 0.85)
- Pay Attention to Small Weights (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
