Research map / Architectures and efficiency / RNNs and normalization
Efficient On-Device Training via Gradient Filtering
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use gradient filtering when finetuning 2 active layers | MobileNetV2 · ImageNet | FLOPs: 113.68M -> 39.27M | Helped |
| Use gradient filtering when finetuning 2 active layers | ResNet-18 · ImageNet | FLOPs: 489.20M -> 28.32M | Helped |
| Use gradient filtering when finetuning 2 active layers | MCUNet · ImageNet | FLOPs: 18.80M -> 6.34M | Helped |
| Use gradient filtering when finetuning 2 active layers | ResNet-34 · ImageNet | accuracy: 69.6 -> 68.6 | Hurt |
| Increase gradient-filter patch size from 2×2 to 4×4 (ablation) | Gradient filtering · ImageNet, MobileNetV2 | accuracy: 63.1 -> 63.4 | Mixed |
| Increase gradient-filter patch size from 2×2 to 4×4 (ablation) | Gradient filtering · ImageNet, ResNet-18 | accuracy: 68.6 -> 68.5 | Hurt |
| Increase gradient-filter patch size from 2×2 to 4×4 (ablation) | Gradient filtering · Augmented Pascal VOC12, UPerNet | mIoU: 47.04 -> 48.00 | Mixed |
| Use gradient filtering rather than vanilla BP | UPerNet, 5 active layers · Augmented Pascal VOC12 | GFLOPs: 503.9 -> 1.97 | Helped |
| Use gradient filtering rather than vanilla BP | UPerNet, 5 active layers · Augmented Pascal VOC12 | mIoU: 47.93 -> 47.04 | Hurt |
| Use gradient filtering rather than vanilla BP | PSPNet, 5 active layers · Augmented Pascal VOC12 | mIoU: 39.54 -> 39.34 | No clear effect |
| Use gradient filtering rather than vanilla BP | PSPNet, 10 active layers · Augmented Pascal VOC12 | mIoU: 53.15 -> 50.88 | Hurt |
| Use gradient filtering rather than vanilla BP | DeepLabV3, 10 active layers · Augmented Pascal VOC12 | GFLOPs: 102.0 -> 2.96 | Helped |
Closest papers by meaning
- Accelerated On-Device Forward Neural Network Training with Module-Wise Descending Asynchronism (NeurIPS 2023 · similarity 0.88)
- Stepping Forward on the Last Mile (NeurIPS 2024 · similarity 0.88)
- E2-Train: Training State-of-the-art CNNs with Over 80% Energy Savings (NeurIPS 2019 · similarity 0.87)
- An Empirical Study of Distributed Deep Learning Training on Edge (Student Abstract) (AAAI 2024 · similarity 0.85)
- Beyond Low-rank Decomposition: A Shortcut Approach for Efficient On-Device Learning (ICML 2025 · similarity 0.84)
- TinyTrain: Resource-Aware Task-Adaptive Sparse Training of DNNs at the Data-Scarce Edge (ICML 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.