Research map / Architectures and efficiency / RNNs and normalization
How Important Is Weight Symmetry in Backpropagation?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace symmetric feedback with fixed random feedback (RndF) | SGD backprop · CIFAR | error rate: 22.73 -> 42.69 | Hurt |
| replace symmetric feedback with fixed random feedback (RndF) | SGD backprop · CIFAR100 | error rate: 55.15 -> 90.88 | Hurt |
| add Batch Normalization to random feedback (ablation) | RndF · CIFAR | error rate: 42.69 -> 24.35 | Helped |
| add Batch Manhattan alone to random feedback (ablation) | RndF · CIFAR | error rate: 42.69 -> 62.71 | Hurt |
| use sign-concordant feedback (uSF) instead of random feedback, with BN+BM (ablation) | RndF+BN+BM · CIFAR100 | error rate: 64.69 -> 53.12 | Helped |
| asymmetric sign-concordant BP (uSF+BN+BM) vs symmetric SGD | SGD · PubFig83-ID | error rate: 63.25 -> 40.67 | Helped |
| asymmetric sign-concordant BP (uSF+BN+BM) vs symmetric SGD | SGD · TIMIT80 | error rate: 23.04 -> 25.12 | Hurt |
| lower sign-concordance probability of random-magnitude feedback (ablation) | brSF/frSF+BN+BM · CIFAR (fixed M,S) | error rate: 19.02 -> 31.54 | Hurt |
| vary Batch Manhattan setting (BM1/BM2/BM3) (ablation) | uSF+BN · CIFAR | error rate: 19.29 / 18.87 / 18.38 | No clear effect |
| freeze last layer, learn lower layers (Bottom) (ablation) | RndF · CIFAR | error rate: 42.69 -> 89.71 | Hurt |
| freeze last layer, learn lower layers (Bottom) (ablation) | uSF+BN+BM · CIFAR | error rate: 19.29 -> 16.72 | Helped |
| add Batch Normalization | SGD · CIFAR | error rate: 22.73 -> 16.75 | Helped |
Closest papers by meaning
- Overcoming Rank Collapse in Feedback Alignment (NeurIPS 2026 · similarity 0.86)
- Improving equilibrium propagation without weight symmetry through Jacobian homeostasis (ICLR 2024 · similarity 0.83)
- The Empirical Impact of Neural Parameter Symmetries, or Lack Thereof (NeurIPS 2024 · similarity 0.82)
- One-Time Soft Alignment Enables Resilient Learning without Weight Transport (NeurIPS 2026 · similarity 0.82)
- Symmetry Induces Structure and Constraint of Learning (ICML 2024 · similarity 0.82)
- Mirror, Mirror of the Flow: How Does Regularization Shape Implicit Bias? (ICML 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.