Research map / Architectures and efficiency / CNN design
Decoupled Greedy Learning of CNNs
Eugene Belilovsky, Michael Eickenberg, Edouard Oyallon
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace CNN-aux with MLP-aux (ablation) | DGL CNN · CIFAR-10 | accuracy: 92.2 -> 90.6 | Hurt |
| Replace CNN-aux with MLP-aux (ablation) | DGL CNN · CIFAR-10 | relative FLOPS: 200% -> 0.7% | Improved |
| Use staged spatial reduction in the auxiliary network instead of CNN-aux (ablation) | DGL CNN · CIFAR-10 | accuracy: 92.2 -> 91.2 | Hurt |
| Use staged spatial reduction in the auxiliary network instead of CNN-aux (ablation) | DGL CNN · CIFAR-10 | relative FLOPS: 200% -> 4.0% | Improved |
| Optimize greedy layers in parallel rather than sequentially (ablation) | DGL · CIFAR-10, 6-layer CNN | accuracy | No clear effect |
| Use DGL rather than DDG for module training | ResNet-110 · CIFAR-10 (K=2) | accuracy: 93.41 vs 93.5±0.1 | No clear effect |
| Use asynchronous scheduling with a slowed layer up to approximately 1.2× (ablation) | Async DGL · CIFAR-10, 5-layer CNN | accuracy: approximately 1.2× | No clear effect |
| Use smaller replay buffers (ablation) | Async DGL · CIFAR-10, 5-layer CNN | accuracy | Hurt |
| Use DGL rather than end-to-end backpropagation | CNNs · CIFAR-10, CNN experiment comparing DNI methods | test accuracy | Improved |
Closest papers by meaning
- SEDONA: Search for Decoupled Neural Networks toward Greedy Block-wise Learning (ICLR 2021 · similarity 0.88)
- Accumulated Decoupled Learning with Gradient Staleness Mitigation for Convolutional Neural Networks (ICML 2021 · similarity 0.86)
- A General Regret Bound of Preconditioned Gradient Method for DNN Training (CVPR 2023 · similarity 0.85)
- Neural Network Optimization Reimagined: Decoupled Techniques for Scratch and Fine-Tuning (TPAMI 2026 · similarity 0.85)
- A Greedy Algorithm for Quantizing Neural Networks (JMLR 2021 · similarity 0.84)
- Catch-Up Mix: Catch-Up Class for Struggling Filters in CNN (AAAI 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
