Research map / Architectures and efficiency / Quantization
On-Device Training Under 256KB Memory
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Quantization-Aware Scaling to int8 SGD-M training (ablation) | MCUNet-5FPS int8 fine-tuning · 8 downstream datasets avg | avg accuracy (%): 64.9 -> 73.5 | Helped |
| use Adam instead of SGD-M for int8 training | MCUNet-5FPS int8 · 8 downstream datasets avg | avg accuracy (%): 64.9 -> 71.8 | Helped |
| use LARS instead of SGD-M for int8 training | MCUNet-5FPS int8 · 8 downstream datasets avg | avg accuracy (%): 64.9 -> 39.8 | Hurt |
| sparse update instead of updating last k layers (ablation) | MbV2/Proxyless/MCUNet transfer learning · 8 downstream datasets avg | avg accuracy at given extra memory: 4.5-7.5x smaller extra memory at higher accuracy | Helped |
| update backbone biases/weights in addition to classifier only (ablation) | transfer learning · 8 downstream datasets avg | avg accuracy | Helped |
| update more biases beyond bias-only plateau (ablation) | bias-only update · 8 downstream datasets avg | avg accuracy | No clear effect |
| add operator reordering and in-place update to sparse update (ablation) | TTE · STM32F746 peak memory | peak memory: 2.4-3.2x reduction | Helped |
| add backward graph pruning for sparse update (ablation) | TTE · STM32F746 peak memory | peak memory: 7-9x reduction | Helped |
| use TTE kernels with sparse update vs TF-Lite Micro full update | MCU training · STM32F746 latency | training speed: 23-25x faster | Helped |
| select updated channels by large magnitude vs small or random (ablation) | sparse tensor update · MCUNet last two blocks, 1/4 weights | accuracy: within 0.2% | No clear effect |
| contribution-analysis-based search for update scheme (ablation) | sparse update · Cars/MCUNet | avg accuracy vs sum of delta acc | Helped |
Closest papers by meaning
- TinyTrain: Resource-Aware Task-Adaptive Sparse Training of DNNs at the Data-Scarce Edge (ICML 2024 · similarity 0.89)
- TinyTTA: Efficient Test-time Adaptation via Early-exit Ensembles on Edge Devices (NeurIPS 2024 · similarity 0.87)
- TinyFoA: Memory Efficient Forward-Only Algorithm for On-Device Learning (AAAI 2025 · similarity 0.85)
- AtomNet: Designing Tiny Models from Operators Under Extreme MCU Constraints (AAAI 2025 · similarity 0.85)
- Stepping Forward on the Last Mile (NeurIPS 2024 · similarity 0.85)
- Beyond Low-rank Decomposition: A Shortcut Approach for Efficient On-Device Learning (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.