Research map / Learning theory and algorithms / Prediction and decision losses
Boosted Trees on a Diet: Compact Models for Resource-Constrained Devices
Nina Herrmann, Jan Stenkamp, Benjamin Karic, Stefan Oehmcke, Fabian Gieseke
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add feature and threshold reuse penalties plus compact bit-wise memory layout to LightGBM training | LightGBM · Covertype (multiclass) | memory to reach 69% accuracy: 16 KB (float32 LightGBM) -> 2 KB | Improved |
| compare ToaD against array-based LightGBM under the same pointer-less layout (ablation) | ToaD · eight datasets, memory-limited | accuracy / R2 | Improved |
| add feature penalty and threshold penalties in training versus no penalty (iota = xi = 0) (ablation) | ToaD · eight datasets, memory-limited | accuracy / R2 | Improved |
| increase feature penalty to 2^12 (ablation) | ToaD · Covertype (256 iterations, depth 2) | accuracy: 35 -> 5 features, about 2% accuracy loss | No clear effect |
| increase feature penalty on datasets with few features (ablation) | ToaD · California Housing and kin8nm | R2 | Hurt |
| raise threshold penalty from smallest to largest (ablation) | ToaD · Covertype (256 iterations, depth 2) | accuracy / number of values: values 1323 -> 18, about 6% accuracy decrease | Mixed |
| set threshold penalty to the maximum 2^15 (ablation) | ToaD · all datasets | model size: global values approach 1 | Hurt |
| moderate threshold penalty near the reuse factor peak (ablation) | ToaD · Wine Quality | reuse factor: ReF above 3 at penalty about 2^8 | Improved |
| increase both penalties jointly (ablation) | ToaD · Covertype, California Housing (256 iterations, depth 2) | memory: about 5 KB -> about 80 Byte | Improved |
| use ToaD layout instead of LightGBM on microcontroller | ToaD · ESP32-S3 prediction runtime | runtime (us): 17.63 -> 137.08 | Hurt |
| use ToaD layout instead of LightGBM on microcontroller | ToaD · Arduino Nano 33 BLE prediction runtime | runtime (us): 102.16 -> 512.89 | Hurt |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 0 of 1 on most shared results.
- LightGBM — LightGBM: A Highly Efficient Gradient Boosting Decision Tree (NeurIPS 2017) · better on 0 of 2 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- HALO: Hardware-Aware Learning to Optimize (ECCV 2020 · similarity 0.86)
- TinyTrain: Resource-Aware Task-Adaptive Sparse Training of DNNs at the Data-Scarce Edge (ICML 2024 · similarity 0.85)
- TinyTTA: Efficient Test-time Adaptation via Early-exit Ensembles on Edge Devices (NeurIPS 2024 · similarity 0.83)
- TinyFoA: Memory Efficient Forward-Only Algorithm for On-Device Learning (AAAI 2025 · similarity 0.83)
- Compressing tree ensembles through Level-wise Optimization and Pruning (ICML 2025 · similarity 0.82)
- Activation Map Compression through Tensor Decomposition for Deep Learning (NeurIPS 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
