Research map / Architectures and efficiency / RNNs and normalization
Scaling Forward Gradient With Local Losses
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use activity perturbation instead of weight perturbation | Forward Gradient · CIFAR-10 supervised, M/8/16 | test error (%): 55.95 -> 59.72 | Helped |
| Add local greedy losses to activity-perturbed forward gradient | FG-A · CIFAR-10 supervised, M/8/16 | test error (%): 59.72 -> 30.68 | Helped |
| Add local greedy losses to activity-perturbed forward gradient | FG-A · ImageNet supervised, L/32/64 | test error (%): 98.83 -> 58.37 | Helped |
| Combine blockwise, patchwise, and groupwise local losses (ablation) | Local Greedy Forward Gradient · CIFAR-10 supervised and ImageNet supervised | test error | Helped |
| Add more channel groups, up to the point where the effect plateaus (ablation) | Local Greedy Forward Gradient · CIFAR-10 supervised, M/8 | test error | Helped |
| Remove stop-gradient from the second view in InfoNCE (ablation) | Perturbation-based contrastive learning · CIFAR-10 contrastive | loss | Hurt |
| Use non-shared perturbations instead of stop-gradient on the second InfoNCE view (ablation) | Perturbation-based contrastive learning · CIFAR-10 contrastive | performance | Hurt |
| Use local layer normalization instead of standard layer normalization (ablation) | LocalMixer · Contrastive learning | test error | Helped |
| Place normalization both before and after each linear layer (ablation) | Forward Gradient LocalMixer · Image classification | training error | Helped |
| Use weaker data augmentation for contrastive learning (ablation) | Forward Gradient and backprop · CIFAR-10 contrastive | performance | No clear effect |
Closest papers by meaning
- The Price of Locality: Why Forward-Forward Underperforms Backpropagation? (NeurIPS 2026 · similarity 0.89)
- Local Loss Optimization in the Infinite Width: Stable Parameterization of Predictive Coding Networks and Target Propagation (ICLR 2025 · similarity 0.88)
- Towards Interpretable Deep Local Learning with Successive Gradient Reconciliation (ICML 2024 · similarity 0.87)
- Forward Learning with Top-Down Feedback: Empirical and Analytical Characterization (ICLR 2024 · similarity 0.86)
- Convolutional Channel-Wise Competitive Learning for the Forward-Forward Algorithm (AAAI 2024 · similarity 0.86)
- FLOPS: Forward Learning with OPtimal Sampling (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.