Research map / Architectures and efficiency / RNNs and normalization
The Tree Ensemble Layer: Differentiability meets Conditional Computation
Hussein Hazimeh, Natalia Ponomareva, Petros Mol, Zhenyu Tan, Rahul Mazumder
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace final dense layers with a dense layer followed by TEL | CNN · CIFAR-10 test | loss: 1.673 ± 0.170 -> 1.202 ± 0.011 | Improved |
| Replace final dense layers with a dense layer followed by TEL | CNN · CIFAR-10 test | number of parameters: 7,548,362 -> 926,465 | Improved |
| Replace final dense layers with a dense layer followed by TEL | CNN · MNIST test | loss: 0.03620 ± 0.00121 -> 0.03379 ± 0.00093 | Improved |
| Replace final dense layers with a dense layer followed by TEL | CNN · Fashion MNIST test | loss: 0.6930 ± 0.0291 -> 0.3247 ± 0.0045 | Improved |
| Replace final dense layers with a dense layer followed by TEL | CNN · Fashion MNIST test | accuracy: 0.9299 ± 0.0012 -> 0.9297 ± 0.0012 | No clear effect |
| Use smooth-step routing instead of logistic routing (ablation) | TEL · PMLB datasets | AUC: 7 wins vs. 7 wins | Mixed |
| Use smooth-step routing instead of logistic routing (ablation) | TEL · PMLB datasets, depth 10 | training speedup: over $10$x | Improved |
| Jointly optimize an ensemble with TEL rather than use stage-wise GBDT | tree ensemble · pima, comparable AUC | number of trees: more than 100 -> 5 | Improved |
| Use TEL instead of GBDT | standalone learner · PMLB datasets | AUC wins: TEL: 12; GBDT: 14 | Mixed |
Closest papers by meaning
- Multistage Defer Trees for Hybrid Interpretability: If at First You Can't Succeed, Tree Again (NeurIPS 2026 · similarity 0.88)
- Neural Tangent Kernels for Axis-Aligned Tree Ensembles (ICML 2024 · similarity 0.87)
- Minimax Rates and Spectral Distillation for Tree Ensembles (NeurIPS 2026 · similarity 0.86)
- Compressing tree ensembles through Level-wise Optimization and Pruning (ICML 2025 · similarity 0.86)
- Tree Ensemble Explainability through the Hoeffding Functional Decomposition and TreeHFD Algorithm (NeurIPS 2025 · similarity 0.86)
- GRANDE: Gradient-Based Decision Tree Ensembles for Tabular Data (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
