Research map / Architectures and efficiency / RNNs and normalization
Weight Feedback Computes the Exact Jacobian Transpose in Modern Deep Networks
Junlong Shen, Xingyu Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use W^T feedback instead of random matrix B (ablation) | WF-Act-PC · VGG-5/CIFAR-10 | test accuracy (%): 27.42 -> 88.25 | Improved |
| add sigma' correction to W^T feedback (ablation) | WF-Act-PC · VGG-5/CIFAR-10 | test accuracy (%): 81.62 -> 88.25 | Improved |
| replace local transport with autograd J^T (ablation) | WF-Act-PC · VGG-5/CIFAR-10 | test accuracy (%): 88.25 -> 89.01 | No clear effect |
| add soft spectral norm clipping (ablation) | WF-Act-PC · VGG-5/CIFAR-10 | test accuracy (%): 87.40 -> 88.25 | Improved |
| increase inner-loop iterations from T=1 to T=10 (ablation) | WF-Act-PC · VGG-5/CIFAR-10 | test accuracy (%): 87.64 -> 88.29 | Improved |
| increase inner-loop iterations from T=20 to T=40 (ablation) | WF-Act-PC · VGG-5/CIFAR-10 | test accuracy (%): 88.25 -> 72.01 | Hurt |
| add normalization gain s correction with BatchNorm (ablation) | WF-Act-PC · BN-VGG-5/CIFAR-10 | test accuracy (%): 86.03 -> 87.60 | Improved |
| add sigma' correction with BatchNorm (ablation) | WF-Act-PC · BN-VGG-5/CIFAR-10 | test accuracy (%): 84.48 -> 87.60 | Improved |
| use exact argmax unpooling instead of nearest-neighbour for MaxPool (ablation) | WF-Act-PC · VGG-5/CIFAR-10 | test accuracy (%): 88.32 -> 89.58 | Improved |
| vary SNC threshold tau in {2,3,5,inf} (ablation) | WF-Act-PC · VGG-5/CIFAR-10 (single seed) | test accuracy (%): 87.9/88.3/87.9/87.4 | No clear effect |
| extend inner loop to T=30 (ablation) | WF-Act-PC · VGG-5/CIFAR-10 (single seed) | test accuracy (%): 88.2 | No clear effect |
| use tuned BP instead of WF-Act-PC (VGG-9) (ablation) | BP (tuned) · CIFAR-100 VGG-9 | test accuracy (%): 71.06 -> 71.90 | Improved |
Closest papers by meaning
- $\mu$PC: Scaling Predictive Coding to 100+ Layer Networks (NeurIPS 2025 · similarity 0.89)
- Only Strict Saddles in the Energy Landscape of Predictive Coding Networks? (NeurIPS 2024 · similarity 0.89)
- One-Time Soft Alignment Enables Resilient Learning without Weight Transport (NeurIPS 2026 · similarity 0.88)
- Benchmarking Predictive Coding Networks -- Made Simple (ICLR 2025 · similarity 0.88)
- Local Loss Optimization in the Infinite Width: Stable Parameterization of Predictive Coding Networks and Target Propagation (ICLR 2025 · similarity 0.87)
- Overcoming Rank Collapse in Feedback Alignment (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
