Research map / Architectures and efficiency / RNNs and normalization
How to Train Your Wide Neural Network Without Backprop: An Input-Weight Alignment Perspective
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Align-ada instead of feedback alignment or direct feedback alignment | Wide CNNs on CIFAR-10 and KMNIST · CIFAR-10 and KMNIST | test accuracy | Helped |
| Use Align-ada instead of gradient descent | CNNs trained on the lowest-data Small CIFAR-10 setting · Small CIFAR-10, lowest data setting | test accuracy | Helped |
| Use Align-ada instead of gradient descent | Wide CNNs · CIFAR-10 and KMNIST | test accuracy | Mixed |
| Replace the changing backpropagation Jacobian with its initialization value and keep activations fixed at initialization (ablation) | Align-zero · CIFAR-10 and KMNIST | test accuracy | Hurt |
| Adapt activations to their current values rather than fixing them at initialization (ablation) | Align-zero, producing Align-ada · CIFAR-10 and KMNIST | test accuracy | Helped |
| Use Align-ada instead of gradient descent | Wide CNNs · ImageNet | training loss | No clear effect |
| Use Align-zero instead of gradient descent | Wide CNNs · ImageNet | training loss | No clear effect |
| Use Align-ada instead of gradient descent | RNNs · Add task | task performance | No clear effect |
| Use Align-ada or Align-zero instead of gradient descent | Wide CNNs · CIFAR-10 and KMNIST, small learning rates | test accuracy | No clear effect |
| Increase network width (ablation) | Align-zero · CIFAR-10 and KMNIST | test accuracy | Helped |
| Increase learning rate (ablation) | Align-ada and Align-zero relative to gradient descent · CIFAR-10 and KMNIST | test accuracy | Hurt |
| Train with randomly permuted weights after training instead of the learned weights (ablation) | Alignment-score evaluation · CIFAR-10 and KMNIST | alignment score | Hurt |
Closest papers by meaning
- Convergence and Alignment of Gradient Descent with Random Backpropagation Weights (NeurIPS 2021 · similarity 0.90)
- Fast Finite Width Neural Tangent Kernel (ICML 2022 · similarity 0.89)
- The Surprising Effectiveness of Infinite-Width NTKs for Characterizing and Improving Model Training (AAAI 2025 · similarity 0.89)
- Just One Layer Norm Guarantees Stable Extrapolation (NeurIPS 2025 · similarity 0.88)
- Better NTK Conditioning: A Free Lunch from (ReLU) Nonlinear Activation in Wide Neural Networks (NeurIPS 2025 · similarity 0.88)
- Forward Learning with Top-Down Feedback: Empirical and Analytical Characterization (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.