Research map / Time series / Causal and generative time series
IncomeSCM: From tabular data set to time-series simulator and causal estimation benchmark
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds IncomeSCM, a sequential structural causal model simulator that fits initial-state mechanisms to the Adult income data and hand-designs transitions, then benchmarks causal effect estimators on a studies-to-income task.
Problem
Causal effect estimators need benchmarks with ground-truth counterfactuals, but existing semi-synthetic and hand-designed simulators are either too simple or lack realistic structure and sequential causal constraints.
Approach
Turn a cross-sectional observational dataset into a discrete-time Markov SCM by fitting initial-state structural equations to real data (random forests, logistic regressions) and composing hand-designed transition mechanisms, yielding complex yet identifiable causal estimation tasks.
Key results
- Best CATE R^2 on Task 1 is 0.30 (T-learner RF); best CATE R^2 for Tasks 2 and 3 is around 0.20 and 0.33.
- IPW estimators return large ATE errors (e.g., IPW RF 57025 vs unbiased ATE ~40919) despite AUC ~0.98 for propensity models.
- Best factual fit (S-learner RF in Task 1) does not yield the best causal effect estimates, showing fit and causal accuracy diverge.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use T-learner (RF) rather than the other tested estimators | IncomeSCM-1.0.CATE Task 1 with full adjustment · IncomeSCM-1.0.CATE Task 1 | CATE R²: 0.30 | Helped |
| Use T-learner (XGB) rather than the other tested estimators | IncomeSCM-1.0.CATE Task 1 with full adjustment · IncomeSCM-1.0.CATE Task 1 | ATE absolute error: 207 | Helped |
| Use DML (Linear) rather than the other tested estimators | IncomeSCM-1.0.CATE Task 2 with minimal adjustment · IncomeSCM-1.0.CATE Task 2 | CATE R²: 0.21 | Helped |
| Use S-learner (RF) rather than the other tested estimators | IncomeSCM-1.0.CATE Task 2 with minimal adjustment · IncomeSCM-1.0.CATE Task 2 | ATE absolute error: 1143 | Helped |
| Use IPW (RF) | IncomeSCM-1.0.CATE Task 1 with full adjustment · IncomeSCM-1.0.CATE Task 1 | ATE absolute error: 57025 | Hurt |
| Use DML (XGB) rather than another CATE estimator | IncomeSCM-1.0.CATE Task 2 with minimal adjustment · IncomeSCM-1.0.CATE Task 2 | CATE R²: -1.13 | Hurt |
| Select methods by factual prediction fit rather than causal-effect performance | IncomeSCM causal effect estimation · IncomeSCM-1.0.CATE | CATE R² | Hurt |
| Use high-performing propensity prediction as a proxy for good ATE estimates | IPW estimators on IncomeSCM-1.0.CATE · IncomeSCM-1.0.CATE | ATE absolute error: 0.98 AUC | Hurt |
| Estimate effects across a six-time-step intervention-to-outcome horizon | IncomeSCM-1.0.CATE · Tasks 1–3 | CATE R²: 0.30, 0.20, and 0.33 | Hurt |
Surprising findings
- Models with comparable factual fit (AUC/R^2) differ greatly in causal effect estimation quality.
- IPW methods with near-perfect propensity discrimination still produce large ATE errors.
Limitations
- Fixed-length trajectories with no censoring or death, so age can exceed realistic limits over long horizons.
- No missing values generated by the simulator.
- Initial-state estimators fit with a single hyperparameter set, not optimized.
Experimental setup
Datasets: Adult
Benchmarks: IncomeSCM-1.0.CATE
Metrics: R^2 CATE absolute error ATE AUC propensity CV R^2 outcome CV
Baselines: IPW IPW-W Matching (EU-NN) S-learner T-learner DML
Builds on: Adult data set Gumbel-Max trick double machine learning backdoor criterion
Closest papers by meaning
- Do Contemporary Causal Inference Models Capture Real-World Heterogeneity? Findings from a Large-Scale Benchmark (ICLR 2025 · similarity 0.87)
- CausalPFN: Amortized Causal Effect Estimation via In-Context Learning (NeurIPS 2025 · similarity 0.86)
- Data Fusion for Partial Identification of Causal Effects (NeurIPS 2025 · similarity 0.85)
- End-To-End Causal Effect Estimation from Unstructured Natural Language Data (NeurIPS 2024 · similarity 0.85)
- Detecting and Measuring Confounding Using Causal Mechanism Shifts (NeurIPS 2024 · similarity 0.85)
- A Practical Approach to Causal Inference over Time (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.