Research map / Probabilistic and scientific modeling / Treatment effects
Causal Contrastive Learning for Counterfactual Regression Over Time
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Causal CPC, a GRU-based encoder pretrained with contrastive predictive coding and an InfoMax term, plus a CLUB-based adversarial balancing decoder, for long-horizon counterfactual regression over time.
Problem
Estimating counterfactual responses over long horizons under time-varying confounding and selection bias, where existing RNN models lack long-term dependency capture and recent transformer models are computationally costly and representations are not invertible.
Approach
Combine CPC (InfoNCE over multiple horizons) to learn long-term history contexts with an InfoMax regularizer that maximizes MI between past and future sub-sequences of the history, and a CLUB-based adversarial treatment-balancing objective, all on a simple GRU backbone.
Key results
- On cancer simulation (gamma=1, sequence length 60) Causal CPC achieves NRMSE 0.83 at tau=1 and 1.26 at tau=10, best across horizons among compared models.
- On semi-synthetic MIMIC-III (sequence length 100), Causal CPC achieves RMSE 0.77 at tau=10 versus 0.88 for CT and 1.49 for G-Net.
- Causal CPC has 8.2k trainable parameters and 4 min prediction time versus 30 min for CT, but does not beat CRN/RMSN at short horizons.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the InfoNCE loss (ablation) | Causal CPC · Cancer simulation, γ=1, τ=1–10 | NRMSE: 1.05 -> 1.07 | Helped |
| add the InfoNCE loss (ablation) | Causal CPC · MIMIC-III, τ=1–10 | NRMSE: 0.62 -> 0.68 | Helped |
| add the InfoMax loss (ablation) | Causal CPC · Cancer simulation, γ=1, τ=1–10 | NRMSE: 1.05 -> 1.13 | Helped |
| add the InfoMax loss (ablation) | Causal CPC · MIMIC-III, τ=1–10 | NRMSE: 0.62 -> 0.74 | Helped |
| use I_CLUB instead of CDC loss (ablation) | Causal CPC · Cancer simulation, γ=1, τ=1–10 | NRMSE: 1.07 -> 1.05 | Helped |
| use I_CLUB instead of CDC loss (ablation) | Causal CPC · MIMIC-III, τ=1–10 | NRMSE: 0.73 -> 0.62 | Helped |
| add representation balancing (ablation) | Causal CPC · Cancer simulation, γ=1, τ=1–10 | NRMSE: 1.08 -> 1.05 | Helped |
| add representation balancing (ablation) | Causal CPC · MIMIC-III, τ=1–10 | NRMSE: 0.69 -> 0.62 | Helped |
| use Causal CPC rather than CT | Causal CPC · MIMIC-III semi-synthetic, sequence length 100, τ=10 | RMSE: 0.88 $ \pm $0.17 -> 0.77$ \pm$ 0.10 | Helped |
| use Causal CPC rather than CRN | Causal CPC · MIMIC-III semi-synthetic, sequence length 100, τ=1 | RMSE: 0.27 $\pm$0.03 -> 0.32$\pm 0.04$ | Hurt |
Surprising findings
- Causal CPC does not outperform baselines in short-term forecasting, and underperforms at the final horizon when trained only to tau=10, which the authors attribute to the contrastive horizon limit.
- Simple GRU with well-designed regularization outperforms the transformer-based Causal Transformer at long horizons.
Limitations
- Weaker short-horizon performance than CRN, RMSN and CT.
- Assumes sequential ignorability; under violation, errors increase for most models.
- Discrete treatments only; continuous extension discussed but not evaluated.
Experimental setup
Datasets: cancer simulation (PK-PD tumor growth) semi-synthetic MIMIC-III
Benchmarks: cancer simulation semi-synthetic MIMIC-III real MIMIC-III
Metrics: NRMSE RMSE training time prediction time
Baselines: MSM RMSN CRN G-Net CT
Builds on: Contrastive Predictive Coding Deep InfoMax CLUB Counterfactual Recurrent Networks Causal Transformer
Closest papers by meaning
- Counterfactual Contrastive Learning with Normalizing Flows for Robust Treatment Effect Estimation (ICML 2025 · similarity 0.88)
- Towards Characterizing Domain Counterfactuals for Invertible Latent Causal Models (ICLR 2024 · similarity 0.88)
- Learning Counterfactual Outcomes Under Rank Preservation (NeurIPS 2025 · similarity 0.88)
- Counterfactual Generative Modeling with Variational Causal Inference (ICLR 2025 · similarity 0.86)
- Observable Neural ODEs for Identifiable Causal Forecasting in Continuous Time (NeurIPS 2026 · similarity 0.86)
- An Empirical Examination of Balancing Strategy for Counterfactual Estimation on Time Series (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.