Research map / Architectures and efficiency / RNNs and normalization
Momentum Auxiliary Network for Supervised Local Learning
Junhao Su, Changpeng Cai, Feiyu Zhu, Chenghao He, Xiaojie Xu, Dongzhi Guan, Chenyang Si
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add EMA of next-block parameters to the auxiliary network (ablation) | DGL, ResNet-32 K=16 · CIFAR-10 | test error (%): 14.08 -> 11.07 | Improved |
| add learnable bias on top of EMA (ablation) | MAN · CIFAR-10 | test error (%): 11.07 -> 9.11 | Improved |
| add EMA on top of learnable bias for ImageNet (ablation) | InfoPro ResNet-101 K=4 · ImageNet | Top-1 error (%): 22.81 -> 21.73 | Improved |
| use next-block parameters directly instead of EMA (ablation) | MAN · CIFAR-10 | test error (%): 14.08 -> 12.93 | Improved |
| use EMA of next-block parameters rather than raw parameters (ablation) | MAN · CIFAR-10 | test error (%): 12.93 -> 11.07 | Improved |
| use next-block parameters directly on ImageNet (ablation) | InfoPro ResNet-101 K=4 · ImageNet | Top-1 error (%): 22.81 -> 22.64 | No clear effect |
| add MAN to InfoPro with K=4 blocks | InfoPro ResNet-101 · ImageNet | Top-1 error (%): 22.81 -> 21.73 | Improved |
| add MAN to DGL | DGL VGG13 K=10 · ImageNet | Top-1 error (%): 35.60 -> 31.99 | Improved |
| add MAN to local learning methods on CIFAR-10 | PredSim, DGL, InfoPro · CIFAR-10 ResNet-32 K=16 | test error (%): 14.08 -> 9.11 | Mixed |
| add MAN to InfoPro for memory | InfoPro ResNet-101 K=4 · ImageNet | GPU memory (GB): 10.37 -> 10.69 | No clear effect |
Closest papers by meaning
- MLAAN: Scaling Supervised Local Learning with Multilaminar Leap Augmented Auxiliary Network (AAAI 2025 · similarity 0.88)
- Scaling Supervised Local Learning with Augmented Auxiliary Networks (ICLR 2024 · similarity 0.87)
- MomentumSMoE: Integrating Momentum into Sparse Mixture of Experts (NeurIPS 2024 · similarity 0.84)
- The Marginal Value of Momentum for Small Learning Rate SGD (ICLR 2024 · similarity 0.84)
- Towards Interpretable Deep Local Learning with Successive Gradient Reconciliation (ICML 2024 · similarity 0.83)
- Regularized Adaptive Momentum Dual Averaging with an Efficient Inexact Subproblem Solver for Training Structured Neural Network (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
