Research map / Architectures and efficiency / RNNs and normalization
Direct Feedback Alignment Scales to Modern Deep Learning Tasks and Architectures
Julien Launay, Iacopo Poli, François Boniface, Florent Krzakala
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train with DFA instead of BP | NeRF · NeRF-Synthetic | PSNR: 31.01 -> 25.41 | Hurt |
| train with DFA instead of BP | NeRF · LLFF-Real | PSNR: 26.50 -> 20.77 | Hurt |
| train with DFA instead of BP | DeepFM · Criteo | AUC: 0.7954 -> 0.7956 | No clear effect |
| train with DFA instead of BP | Deep&Cross · Criteo | AUC: 0.8104 -> 0.8009 | Hurt |
| train with DFA instead of BP | AFN · Criteo | AUC: 0.7933 -> 0.7924 | No clear effect |
| train with DFA instead of BP | GraphConv · Cora | accuracy (%): 80.1 -> 79.9 | No clear effect |
| train with DFA instead of BP | ChebConv · Cora | accuracy (%): 79.2 -> 75.4 | Hurt |
| train with DFA instead of BP | DNAConv · PubMed | accuracy (%): 87.2 -> 79.9 | Hurt |
| DFA instead of shallow training (top layer only) | Deep&Cross · Criteo | AUC: 0.7324 -> 0.8009 | Improved |
| use Adam instead of baseline optimizer (DFA, Macro) (ablation) | Transformer · WikiText-103 | validation perplexity: 95.0 -> 77.1 | Improved |
| increase beta2 from 0.98 to 0.999 (DFA, Macro) (ablation) | Transformer · WikiText-103 | validation perplexity: 77.1 -> 55.0 | Improved |
| add plateau LR scheduler (DFA, Macro) (ablation) | Transformer · WikiText-103 | validation perplexity: 55.0 -> 52.0 | Improved |
Closest papers by meaning
- Overcoming Rank Collapse in Feedback Alignment (NeurIPS 2026 · similarity 0.89)
- One-Time Soft Alignment Enables Resilient Learning without Weight Transport (NeurIPS 2026 · similarity 0.89)
- Implicit Regularization in Feedback Alignment Learning Mechanisms for Neural Networks (ICML 2024 · similarity 0.87)
- Forward Learning with Top-Down Feedback: Empirical and Analytical Characterization (ICLR 2024 · similarity 0.86)
- DFA-GNN: Forward Learning of Graph Neural Networks by Direct Feedback Alignment (NeurIPS 2024 · similarity 0.86)
- Exploring and Exploiting the Asymmetric Valley of Deep Neural Networks (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
