Research map / Learning theory and algorithms / Prediction and decision losses
Discretionary Trees: Understanding Street-Level Bureaucracy via Machine Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Uses short decision trees (SER-DT) and XGBoost to show homelessness caseworker assignments are highly predictable, and that the residual discretionary decisions target less vulnerable households and yield marginal benefit gains when moving households to more intensive interventions.
Problem
Understanding how street-level bureaucrats (homelessness caseworkers) make intervention-assignment decisions: whether they follow simple heuristics, whether their decisions are consistent, and what the cost/benefit consequences of their discretion are.
Approach
Treat caseworker assignments as a prediction problem: learn short interpretable decision trees (SER-DT) to capture heuristic rules, label mismatches with those rules as discretion, and evaluate discretionary subgroups on vulnerability scores and counterfactual marginal benefit estimated from BART models.
Key results
- Short SER-DT trees reach AUC 0.87 for Transitional Housing and 0.76 for Emergency Shelter, while XGBoost reaches 0.94 (TH) and 0.95 (ES), indicating high caseworker consistency beyond simple rules.
- Discretionary reassignments (predicted ES to TH, and predicted TH to ES) are concentrated among households with significantly lower vulnerability scores than random comparison groups.
- The predicted-ES-to-TH discretionary group has significantly higher marginal benefit than randomly chosen comparison groups, while the TH-to-ES group shows no significant marginal benefit loss.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use XGBoost instead of short SER-DT trees | Intervention assignment prediction · Emergency Shelter assignment | AUC: 0.7671 -> 0.9452 | Helped |
| Use XGBoost instead of short SER-DT trees | Intervention assignment prediction · Transitional Housing assignment | AUC: 0.8766 -> 0.9434 | Helped |
| Apply discretion to assign ES-predicted households to TH | Caseworker intervention decisions · EStoTH households | Marginal benefit of TH rather than ES | Helped |
| Apply discretion to assign TH-predicted households to ES | Caseworker intervention decisions · THtoES households | Marginal benefit of TH rather than ES | No clear effect |
| Apply discretionary assignments to households with lower vulnerability scores | Caseworker intervention decisions · EStoTH and THtoES households | Vulnerability score | Mixed |
Surprising findings
- Caseworkers' discretionary reassignments to more intensive intervention yield larger marginal benefits, despite caseworkers having no access to outcome predictions.
- Discretion is applied mainly to less vulnerable households, the opposite of what a vulnerability-first prioritization would predict.
Limitations
- Data come from a single metropolitan area (St. Louis) and a pre-2014 period, limiting generalizability.
- Vulnerability scores are a self-constructed approximation of VI-SPDAT since direct scores were unavailable.
- Marginal benefits rely on counterfactual machine learning estimates rather than observed outcomes.
Experimental setup
Datasets: St. Louis HMIS administrative records (2007-2014)
Benchmarks: St. Louis HMIS homelessness intervention assignments
Metrics: AUC 95% confidence interval (DeLong's method) Spearman rank correlation vulnerability score comparison marginal benefit (probability of return to homelessness)
Baselines: CART XGBoost
Builds on: SER-DT XGBoost Bayesian Additive Regression Trees CART DeLong's method
Closest papers by meaning
- A Bayesian Spatial Model to Correct Under-Reporting in Urban Crowdsourcing (AAAI 2024 · similarity 0.82)
- The Value of Prediction in Identifying the Worst-Off (ICML 2025 · similarity 0.82)
- Evaluating Index-based Treatment Allocation in Underresourced Communities (AAAI 2025 · similarity 0.82)
- Allocation Requires Prediction Only if Inequality Is Low (ICML 2024 · similarity 0.82)
- Predictive Modeling of Homeless Service Assignment: A Representation Learning Approach (AAAI 2025 · similarity 0.82)
- Providing Fair Recourse over Plausible Groups (AAAI 2024 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.