Research map / Probabilistic and scientific modeling / Treatment effects
Transformer-Based Spatial-Temporal Counterfactual Outcomes Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a Transformer- and CNN-based estimator for counterfactual outcomes of spatial-temporal point-process data under stochastic treatment interventions, with consistency and asymptotic normality results, and applies it to conflicts and forest loss in Colombia.
Problem
Estimating counterfactual outcomes for spatial-temporal data with high-dimensional binary treatments and outcomes (point patterns), where classical statistical methods have limited performance and generalization and propensity scores for high-dimensional treatments are hard to compute.
Approach
Inverse probability weighting over spatial Poisson point processes, where high-dimensional treatments are reduced to their count of treated locations so propensity scores and counterfactual probabilities are obtained from CNN-regressed Poisson rates, and a Transformer with a variational/MLE objective estimates outcome intensity functions; it also generalizes treatment interventions to time-varying
Key results
- On synthetic data with M=1, the proposed estimator achieves lower RER than MSMs, RMSNs, Causal Forest and LR across c in {3..7} and time lengths 32/48/64 (e.g., T=48, c=3: 0.062 vs 0.081 for MSMs).
- Real-data estimates of forest loss rise with both conflict duration M and intensity c (e.g., M=1 about 20.5-20.8 vs M=7 reaching 24.7-28.0), supporting the claim that longer and more intense conflicts increase forest loss.
- Replacing the Transformer with RNNs and relaxing the Poisson assumption (Gaussian kernel) are reported to reduce performance only modestly/robustly, per ablations.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase conflict intervention duration M | Ours · Colombia forest loss | estimated number of forest loss events | Helped |
| Increase conflict intervention intensity c | Ours · Colombia forest loss | estimated number of forest loss events | Mixed |
| Replace Transformer with RNNs for intensity estimation (ablation) | Ours · Synthetic data, T=64, M=1 | relative error rate (RER) | Hurt |
| Relax the Poisson assumption by adding a Gaussian kernel to synthetic intensity functions (ablation) | Ours · Synthetic data, T=64, M=1 | relative error rate (RER) | No clear effect |
Surprising findings
- The real-data table reports the forest-loss estimate at M=1 as nearly constant across c, with the increase appearing mainly with larger M.
Limitations
- Restricted to discrete point-process outcomes and binary treatments; more general spatial-temporal stochastic processes are left to future work.
- Baselines had to be adapted by transforming high-dimensional synthetic data into scalar series.
- Real-data validation relies on consistency with prior literature rather than known ground-truth counterfactuals.
Experimental setup
Datasets: Global Forest Change data (Landsat-derived forest loss) UCDP Georeferenced Event Dataset synthetic spatial Poisson point process data
Benchmarks: synthetic spatial-temporal Poisson point process data
Metrics: relative error rate (RER) estimated forest loss count
Baselines: MSMs RMSNs Causal Forest LR
Builds on: marginal structural models inverse probability weighting variational autoencoder
Closest papers by meaning
- GST-UNet: A Neural Framework for Spatiotemporal Causal Inference with Time-Varying Confounding (NeurIPS 2025 · similarity 0.83)
- Learning Counterfactual Outcomes Under Rank Preservation (NeurIPS 2025 · similarity 0.83)
- Towards Characterizing Domain Counterfactuals for Invertible Latent Causal Models (ICLR 2024 · similarity 0.82)
- Smooth Sailing: Lipschitz-Driven Uncertainty Quantification for Spatial Associations (NeurIPS 2025 · similarity 0.82)
- SpaCE: The Spatial Confounding Environment (ICLR 2024 · similarity 0.82)
- An Empirical Examination of Balancing Strategy for Counterfactual Estimation on Time Series (ICML 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.