Research map / Learning theory and algorithms / Prediction and decision losses
Empowering Decision Trees via Shape Function Branching
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Shape Generalized Trees (SGTs), decision trees whose internal nodes apply learnable axis-aligned shape functions (plus bivariate and multi-way variants) and a ShapeCART induction algorithm, showing smaller trees with higher accuracy than linear trees on 26 tabular datasets.
Problem
Axis-aligned linear decision trees must repeatedly split on the same feature to capture non-linear feature effects, producing deep, large trees that are harder to interpret.
Approach
Replace each linear threshold split with a learnable axis-aligned shape function on one feature (or a bivariate / multi-way shape function), learned in each node by binning the feature with an internal CART/BiCART tree and then assigning bins to branches via coordinate descent, with an optional TAO refinement step.
Key results
- SGT$_3$-C reaches 87.8% best average test accuracy among axis-aligned TDIDT methods vs 85.2% for CART on the same datasets.
- S$^2$GT-C achieves 91.4% best average accuracy vs 89.6% for BiCART in bivariate comparisons.
- TAO-refined SGT$_3$-T achieves 88.0% best average accuracy, the highest among axis-aligned approaches.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use shape-function branching instead of axis-aligned threshold splits | Decision trees · Average test accuracy across datasets, axis-aligned approaches | accuracy (%): 85.2 -> 86.3 | Helped |
| Use ternary rather than binary shape-function branching | SGT-C · Average test accuracy across datasets, axis-aligned approaches | accuracy (%): 86.3 -> 87.8 | Helped |
| Apply TAO refinement | SGT-C · Average test accuracy across datasets, axis-aligned approaches | accuracy (%): 86.3 -> 86.8 | Helped |
| Use bivariate shape-function branching instead of bivariate linear splits | Bivariate trees · Average test accuracy across datasets, bivariate approaches | accuracy (%): 89.6 -> 91.4 | Helped |
| Use ternary rather than binary bivariate shape-function branching | S$^2$GT-C · Average test accuracy across datasets, bivariate approaches | accuracy (%): 91.4 -> 91.7 | Helped |
| Apply TAO refinement | S$^2$GT-C · Average test accuracy across datasets, bivariate approaches | accuracy (%): 91.4 -> 91.6 | Helped |
Surprising findings
- S$^2$GT-C at depth 2 matches BiTAO and BiCART at depth 6 on eye-movements and electricity, indicating strong size compression.
Limitations
- Restricted to tabular data, inheriting tree limitations
- Shape functions can be harder to interpret than axis-aligned splits; no human-subject interpretability study
- Bivariate and multi-way variants are memory- and time-intensive; BiCART/BiTAO exceeded 32GB on Covertype, Eucalyptus and Higgs
Experimental setup
Datasets: adult avila bank bean bidding covtype electricity eucalyptus eye-movements eye-state fault gas-drift
Benchmarks: 26 real-world tabular classification datasets
Metrics: average test accuracy accuracy per depth
Baselines: CART SERDT HSTree AxTAO DPDT SPLIT BiCART BiTAO
Builds on: CART TAO BiCART GAMs
Closest papers by meaning
- Literati: Towards Anytime Optimal Shape Generalized Trees via AO* (NeurIPS 2026 · similarity 0.93)
- TREE-G: Decision Trees Contesting Graph Neural Networks (AAAI 2024 · similarity 0.85)
- Learning Small Decision Trees with Few Outliers: A Parameterized Perspective (AAAI 2024 · similarity 0.85)
- Leveraging Predictive Equivalence in Decision Trees (ICML 2025 · similarity 0.84)
- Branches: Efficiently Seeking Optimal Sparse Decision Trees via AO* (ICML 2025 · similarity 0.84)
- GradTree: Learning Axis-Aligned Decision Trees with Gradient Descent (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.