Research map / Learning theory and algorithms / Prediction and decision losses

Decision-aware Training of Spatiotemporal Forecasting Models to Select a Top-K Subset of Sites for Intervention

ICML 2025 · Learning theory and algorithms › Prediction and decision losses · approach: Adaptive testing and experiments

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a ratio-estimator ranking and decision-aware training (direct BPR via perturbed optimizers and a likelihood-plus-BPR-constraint objective, DAML) for top-K site selection, evaluated on synthetic data, opioid overdoses, and whooping cranes.

Problem

Choosing a top-K subset of spatial sites for intervention using probabilistic spatiotemporal forecasts, where the fraction of best possible reach (BPR) is the decision metric; open questions are how to rank sites given a model and how to train the model to maximize BPR without losing forecast qualit

Approach

Rank sites by the expected normalized count E[y/(1·y)] (the ratio estimator), which is a tighter bound on BPR than the per-site mean; train with score-function gradients plus perturbed top-K optimizers to make BPR differentiable, and combine likelihood with a BPR constraint (DAML) via a penalty method.

Key results

• On the ANWR cranes test set (2009-2010), DAML improved BPR-50 by up to 0.05 over ML training with modest likelihood decay, and BPR-only reached 0.41 versus 0.38 for NLL only.
• On Cook County (test 2021-2022), BPR-only training improved BPR-100 by about +0.04 over ML, while DAML kept BPR at 0.80 vs 0.78 for NLL only with no visible likelihood decay.
• In a toy 9-site example, the ratio estimator achieved BPR at K=3 of 0.625 versus 0.231 for the per-site mean.

Measured effects reported in the paper

Surprising findings

• The per-site mean ranking can select very different sites than the ratio estimator even when the true data-generating model is known, yielding BPR differences over 0.4 at K=1 in a toy example.
• Direct BPR optimization can produce nonsensical likelihood values, and surrogate methods (PG, SPO+) gave lower-quality decisions than DAML on the overdose and crane tasks.

Limitations

• Only one type of model misspecification considered per task and a single model family compared, without wide model comparison.
• Fixed K values; no sensitivity analysis to K or other hyperparameters.
• Focusing on BPR alone may not capture all utility; site selection assumes larger outcome means better intervention target.

Experimental setup

Datasets: MA Fatal Overdoses Cook County IL Fatal Overdoses ANWR TX Cranes synthetic quantized Gaussian data

Benchmarks: MA Fatal Overdoses Cook County IL Fatal Overdoses ANWR TX Cranes

Metrics: BPR held-out log-likelihood MAE RMSE

Baselines: ML estimation (NLL only) BPR only PG SPO+ EpiGNN last timestep avg over history chance decision

Builds on: perturbed optimizers score function estimator SPO+ perturbed gradient direct loss minimization

Closest papers by meaning

• GST-UNet: A Neural Framework for Spatiotemporal Causal Inference with Time-Varying Confounding (NeurIPS 2025 · similarity 0.84)
• MOVEBENCH: A Benchmark for Global-Scale Wildlife Movement Forecasting (NeurIPS 2026 · similarity 0.82)
• SPATIALEPIBENCH: Benchmarking Spatial Information and Epidemic Priors in Forecasting (NeurIPS 2026 · similarity 0.82)
• Context in Public Health for Underserved Communities: A Bayesian Approach to Online Restless Bandits (AAAI 2025 · similarity 0.82)
• Beyond the Training Distribution: Evaluating Predictions Under Distribution Shift and Selection Bias (NeurIPS 2026 · similarity 0.82)
• FireScope: Wildfire Risk Raster Prediction With a Chain-of-Thought Oracle (CVPR 2026 · similarity 0.81)