Research map / Probabilistic and scientific modeling / Treatment effects
Tailoring Strictly Proper Scoring Rules for Downstream Tasks: An Application to Causal Inference
Roman Plaud, Alexandre Perez-Lebel, Antoine Saillenfest, Thomas Bonald, Marine Le Morvan, Gaël Varoquaux, Matthieu Labeau
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace log-loss with the tailored IPW loss and canonical mapping | IPW estimator with linear propensity model · Kang & Schafer | RMSE: 66.90 -> 2.68 | Improved |
| replace log-loss with the tailored loss | AIPW with linear propensity model · Kang & Schafer | RMSE: 46.51 -> 5.07 | Improved |
| replace log-loss with the tailored loss | MLP propensity model · ACIC 2017 | RMSE: better in 95 out of 96 configurations | Improved |
| replace log-loss with the tailored loss | XGBoost propensity model · ACIC 2017 | RMSE: better in 62 out of 96 cases | Mixed |
| high selection strength (propensities near 0/1) versus low strength | tailored loss vs log-loss · ACIC 2017 | relative RMSE improvement | Improved |
| add the variance (1/p^3) term to the bias-only loss (ablation) | tailored loss · Kang & Schafer | RMSE: IPW 3.39 -> 2.68 | Improved |
| add the variance term to the bias-only loss (ablation) | tailored loss · IHDP and Jobs | RMSE: IHDP IPW 0.27 -> 0.34 | No clear effect |
| pair tailored loss with its canonical quartic mapping instead of sigmoid (ablation) | logistic regression with SGD · Kang & Schafer simulation | max gradient norm: 1024 -> 0.275 | Improved |
| use the analytical p-y backward pass instead of autograd (ablation) | MLP with tailored loss · synthetic N=10^6, d=100 | relative training time: 1.14x -> 1.02x | Improved |
| use quartic canonical mapping in XGBoost custom objective | XGBoost · synthetic N=10^6, d=100 | relative training time: 1.00x -> 5.03x | Hurt |
| use tailored loss instead of log-loss, Jobs dataset | IPW with linear propensity model · Jobs | RMSE: 0.46 -> 0.03 | Improved |
| post-hoc trimming versus tailored loss | IPW baseline · Jobs | mean rank: Trimming 1.70 vs Custom 1.60 (IPW) | No clear effect |
Closest papers by meaning
- Inverse-Variance Weighting for Estimation of Heterogeneous Treatment Effects (ICML 2024 · similarity 0.88)
- An effective framework for estimating individualized treatment rules (NeurIPS 2024 · similarity 0.87)
- Optimistic Algorithms for Adaptive Estimation of the Average Treatment Effect (ICML 2025 · similarity 0.87)
- Generalized Bayes for Causal Inference (NeurIPS 2026 · similarity 0.87)
- Collaborative Heterogeneous Causal Inference Beyond Meta-analysis (ICML 2024 · similarity 0.87)
- ABC3: Active Bayesian Causal Inference with Cohn Criteria in Randomized Experiments (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
