Research map / Architectures and efficiency / Pruning and efficient tuning
Enabling Fine-Tuning of Direct Feedback Alignment via Feedback-Weight Matching
Yunseok Lee, Seulki Lee
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply feedback-weight matching before DFA fine-tuning | 6-layer FC network · SVHN (from CIFAR-100) | test accuracy: 74.70 -> 82.67 | Improved |
| apply feedback-weight matching before DFA fine-tuning | BERT-Small · GLUE STSB | Pearson correlation: 0.10 -> 0.76 | Improved |
| apply feedback-weight matching before DFA fine-tuning | BERT-Small · GLUE CoLA | Matthews correlation: 0.06 -> 0.53 | Improved |
| apply feedback-weight matching before DFA fine-tuning | ViT-Small · ImageNette | accuracy: 0.210 -> 0.319 | Improved |
| apply feedback-weight matching before DFA fine-tuning | 4-layer FC network · CIFAR-10 (from TinyImageNet) | test accuracy: 56.75 -> 55.51 | Mixed |
| apply feedback-weight matching before DFA fine-tuning | BERT-Small · GLUE QQP | accuracy: 67.0 -> 65.8 | Hurt |
| add weight matching (re-initialize weights from feedback) (ablation) | DFA fine-tuning · SVHN (4-layer, from TinyImageNet) | test accuracy: 79.77 -> 83.16 | Improved |
| add feedback matching (decompose pre-trained weights) (ablation) | DFA fine-tuning · CIFAR-10 (6-layer, from TinyImageNet) | test accuracy: 55.03 -> 55.54 | Improved |
| add weight decay on top of feedback-weight matching (ablation) | 4-layer FC network · CIFAR-10 (from CIFAR-100) | test accuracy: 48.82 -> 55.38 | Improved |
| add weight decay to standard DFA without matching (ablation) | standard DFA fine-tuning · CIFAR-10 (4-layer, from TinyImageNet) | test accuracy | No clear effect |
| train feedback matrices 3 epochs instead of 1 (ablation) | feedback matching · CIFAR-10 (4-layer, from CIFAR-100) | test accuracy: 51.38 -> 55.38 | Improved |
| apply strong weight decay (1e-2) to feedback-weight matching (ablation) | 4-layer FC network · CIFAR-10 (from CIFAR-100) | train/test accuracy | Hurt |
Closest papers by meaning
- Overcoming Rank Collapse in Feedback Alignment (NeurIPS 2026 · similarity 0.87)
- DFA-GNN: Forward Learning of Graph Neural Networks by Direct Feedback Alignment (NeurIPS 2024 · similarity 0.86)
- One-Time Soft Alignment Enables Resilient Learning without Weight Transport (NeurIPS 2026 · similarity 0.86)
- Implicit Regularization in Feedback Alignment Learning Mechanisms for Neural Networks (ICML 2024 · similarity 0.86)
- Gradient-Weight Alignment as a Train-Time Proxy for Generalization in Classification Tasks (NeurIPS 2025 · similarity 0.86)
- Domain-Aware Fine-Tuning: Enhancing Neural Network Adaptability (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
