Research map / Probabilistic and scientific modeling / Causal discovery
Counterfactually Comparing Abstaining Classifiers
Yo Joong Choe, Aditya Gangrade, Aaditya Ramdas
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use doubly robust estimator instead of plug-in with super learner | counterfactual score CI · simulated MAR | CI miscoverage: 0.91 -> 0.05 | Improved |
| use DR instead of IPW with random forest nuisance | counterfactual score CI · simulated MAR | CI miscoverage: 0.14 -> 0.05 | Improved |
| use DR instead of IPW with super learner | counterfactual score CI · simulated MAR | CI width: 0.12 -> 0.06 | Improved |
| IPW vs DR with super learner | counterfactual score CI · simulated MAR | CI miscoverage: 0.03 vs 0.05 | No clear effect |
| use linear/logistic nuisance estimators for DR (ablation) | DR estimator · simulated MAR | CI miscoverage: 0.05 -> 1.00 | Hurt |
| use random forest instead of linear nuisance estimators for DR (ablation) | DR estimator · simulated MAR | CI miscoverage: 1.00 -> 0.05 | Improved |
| use random forest instead of super learner for plug-in (ablation) | plug-in estimator · simulated MAR | CI miscoverage: 0.91 -> 0.64 | Improved |
| increase sample size or absolute score difference | DR-CI hypothesis test · simulated MAR | power: power reaches 1 | Improved |
| compare classifiers differing only in abstention threshold (0.8 vs 0.5) | DR CI · CIFAR-100 scenario I | counterfactual score difference: (-0.005, 0.018) | No clear effect |
| compare classifiers with different stochastic abstention rules (SR vs Gini) | DR CI · CIFAR-100 scenario II | counterfactual score difference: (-0.014, 0.008) | No clear effect |
Closest papers by meaning
- Learning Counterfactual Outcomes Under Rank Preservation (NeurIPS 2025 · similarity 0.84)
- Mind the Gap: A Causal Perspective on Bias Amplification in Prediction & Decision-Making (NeurIPS 2024 · similarity 0.83)
- Counterfactual Fairness by Combining Factual and Counterfactual Predictions (NeurIPS 2024 · similarity 0.83)
- Trustworthy Actionable Perturbations (ICML 2024 · similarity 0.83)
- Arbitrariness and Social Prediction: The Confounding Role of Variance in Fair Classification (AAAI 2024 · similarity 0.83)
- Fair Classification with Partial Feedback: An Exploration-Based Data Collection Approach (ICML 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
