Research map / Architectures and efficiency / CNN design
Relay Backpropagation for Effective Learning of Deep Convolutional Neural Networks
Li Shen, Zhouchen Lin, Qingming Huang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use relay gradient limit instead of standard BP | Model A · Places2 val (single crop) | top-5 error: 19.00 -> 17.83 | Improved |
| use relay gradient limit instead of standard BP | Model B · Places2 val (single crop) | top-5 error: 18.69 -> 17.86 | Improved |
| add auxiliary loss with standard BP (multi-loss) (ablation) | Model A · Places2 val (single crop) | top-5 error: 19.00 -> 18.84 | No clear effect |
| add auxiliary loss with standard BP (multi-loss) (ablation) | Model B · Places2 val (single crop) | top-5 error: 18.69 -> 18.68 | No clear effect |
| limit each loss's gradient to its own segments (Relay BP) versus multi-loss with standard BP (ablation) | Model A · Places2 val (single crop) | top-5 error: 18.84 -> 17.83 | Improved |
| limit each loss's gradient to its own segments (Relay BP) versus multi-loss with standard BP (ablation) | Model B · Places2 val (single crop) | top-5 error: 18.68 -> 17.86 | Improved |
| use relay gradient limit instead of standard BP | Model A · Places2 val (single model) | top-5 error: 17.19 -> 16.33 | Improved |
| use relay gradient limit instead of standard BP | Model B · Places2 val (single model) | top-5 error: 16.89 -> 16.36 | Improved |
| use relay gradient limit instead of standard BP | ResNet-50 · ImageNet 2012 val | top-5 error: 5.37 -> 4.93 | Improved |
| use relay gradient limit instead of standard BP | ResNet-50 · ImageNet 2012 val | top-1 error: 21.17 -> 20.26 | Improved |
| use relay gradient limit instead of standard BP | Inception-v3 · ImageNet 2012 val | top-5 error: 4.43 -> 3.97 | Improved |
| use relay gradient limit instead of standard BP | Inception-v3 · ImageNet 2012 val | top-1 error: 19.18 -> 18.52 | Improved |
Closest papers by meaning
- Refining Architectures of Deep Convolutional Neural Networks (CVPR 2016 · similarity 0.86)
- Backpropagation-Free Deep Learning with Recursive Local Representation Alignment (AAAI 2023 · similarity 0.86)
- i-RevNet: Deep Invertible Networks (ICLR 2018 · similarity 0.86)
- The Reversible Residual Network: Backpropagation Without Storing Activations (NeurIPS 2017 · similarity 0.85)
- Accelerated training through iterative gradient propagation along the residual path (ICLR 2025 · similarity 0.84)
- Convolutional Channel-Wise Competitive Learning for the Forward-Forward Algorithm (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
