Research map / Time series / Causal and generative time series
An Empirical Examination of Balancing Strategy for Counterfactual Estimation on Time Series
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Empirical study showing that representation-based balancing modules (CRN's gradient reversal, Causal Transformer's domain confusion, PCB contrastive balancing) do not help and often hurt multi-step temporal counterfactual estimation across Tumor, MIMIC-III, and synthetic settings, including cold-start and distribution-shift scenarios.
Problem
Whether balancing strategies that reduce covariate disparity between treatment groups actually improve counterfactual outcome estimation on time series, where they are widely assumed to be effective.
Approach
Systematically re-evaluates balanced versus ERM (no balancing) variants of temporal models across varying treatment-bias strengths, cold-start truncation, and distribution shift, and diagnoses failures via covariate distribution analysis, reconstruction loss of representations, and representation visualization.
Key results
- On the Tumor dataset at gamma=10, CT w/o BRM achieves RMSE 4.395 at tau=1 versus 5.746 for CT with BRM, and CRN w/o BRM outperforms CRN at longer horizons.
- PCB balancing is the worst variant: CRN-PCB reaches RMSE 10.617 at tau=4 at gamma=10 versus 7.690 for CRN w/o BRM.
- On M5 factual estimation, ERM variants generally outperform balanced variants across horizons 1 to 6.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add PCB balancing to Causal Transformer (ablation) | Causal Transformer · Tumor Growth, standard supervised learning, γ=1–10, τ=5 | RMSE: 0.832±0.059 -> 1.033±0.117; 1.043±0.145 -> 2.290±0.701; 7.493±2.829 -> 7.132±3.479 | Hurt |
| Add PCB balancing to CRN (ablation) | CRN · Tumor Growth, standard supervised learning, γ=1–10, τ=5 | RMSE: 0.817±0.041 -> 0.992±0.120; 1.690±0.641 -> 1.599±0.350; 7.326±3.210 -> 10.304±3.394 | Hurt |
| Add the balanced representation module (ablation) | Causal Transformer · Tumor Growth, standard supervised learning, γ=1, τ=5 | RMSE: 0.808±0.051 -> 0.832±0.059 | Hurt |
| Add the balanced representation module (ablation) | CRN · Tumor Growth, standard supervised learning, γ=1, τ=5 | RMSE: 0.787±0.030 -> 0.817±0.041 | Hurt |
| Add the balanced representation module (ablation) | Causal Transformer · Tumor Growth, standard supervised learning, γ=3, τ=5 | RMSE: 0.965±0.142 -> 1.043±0.145 | Hurt |
| Add the balanced representation module (ablation) | CRN · Tumor Growth, standard supervised learning, γ=3, τ=5 | RMSE: 0.957±0.129 -> 1.690±0.641 | Hurt |
| Add the balanced representation module (ablation) | Causal Transformer · Tumor Growth, standard supervised learning, γ=10, τ=5 | RMSE: 5.080±1.736 -> 7.493±2.829 | Hurt |
| Add the balanced representation module (ablation) | CRN · Tumor Growth, standard supervised learning, γ=10, τ=5 | RMSE: 7.727±2.093 -> 7.326±3.210 | Helped |
| Add the balanced representation module (ablation) | Causal Transformer and CRN · MIMIC-III, standard supervised learning, τ=6 | RMSE: 0.75±0.23 -> 0.79±0.29; 1.01±0.49 -> 1.13±0.54 | Mixed |
| Add the balanced representation module (ablation) | Causal Transformer and CRN · Tumor Growth, short-term history cold-start | RMSE | Mixed |
| Add the balanced representation module (ablation) | Causal Transformer and CRN · Tumor Growth, distribution-shift cold-start | RMSE | Hurt |
| Use sequential models instead of non-sequential models | Causal Transformer and CRN · Tumor Growth, treatment bias varying from γ=0 to γ=10 | RMSE | Mixed |
Surprising findings
- Balanced representation modules fail to improve and often worsen counterfactual RMSE in multi-step prediction, even under significant treatment bias.
- Balanced representations show high reconstruction error at high treatment bias, indicating loss of covariate information.
Limitations
- Evaluation restricted to representation-based balancing methods (AGR, CDC, PCB) and a few sequential models.
- Main results rely on synthetic and semi-synthetic data where counterfactuals are simulated.
- M5 results are factual only, lacking counterfactual ground truth.
Experimental setup
Datasets: Tumor Growth simulator MIMIC-III (semi-synthetic) M5 (factual only) pure synthetic autoregressive data
Benchmarks: Tumor Growth simulator MIMIC-III semi-synthetic M5
Metrics: RMSE
Baselines: Causal Transformer CRN RMSN G-Net MSM CFR TARnet OLS Random
Builds on: Counterfactual Recurrent Network Causal Transformer Counterfactual Regression Recurrent Marginal Structural Networks G-Net
Closest papers by meaning
- Counterfactual Contrastive Learning with Normalizing Flows for Robust Treatment Effect Estimation (ICML 2025 · similarity 0.87)
- Learning Counterfactual Outcomes Under Rank Preservation (NeurIPS 2025 · similarity 0.87)
- Towards Characterizing Domain Counterfactuals for Invertible Latent Causal Models (ICLR 2024 · similarity 0.86)
- Causal Contrastive Learning for Counterfactual Regression Over Time (NeurIPS 2024 · similarity 0.86)
- A Dual-module Framework for Counterfactual Estimation over Time (ICML 2024 · similarity 0.86)
- Treatment Effects Estimation By Uniform Transformer (ICLR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.