Research map / Label-efficient and robust learning / Recognition and long-tail
From Biased Selective Labels to Pseudo-Labels: An Expectation-Maximization Framework for Learning from Biased Decisions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DCEM, an EM-based method that learns from selective labels under disparate censorship by treating untested outcomes as latent and regularizing predictions with a learned testing propensity, evaluated on synthetic data and MIMIC-III sepsis.
Problem
Learning classifiers when labels are observed only for tested individuals and untested individuals are imputed as negative, with testing decisions biased across subgroups (disparate censorship), which can amplify bias against disadvantaged groups.
Approach
Cast disparate censorship as a latent-variable problem and derive an EM procedure: the E-step imputes soft pseudo-labels for untested individuals, and the M-step adds a causal regularizer that pushes predictions down where the estimated testing probability is high, counterbalancing labeling bias while leaving tested individuals fully supervised.
Key results
- On synthetic data at q_y=0.5, k=1, q_t=2, DCEM achieves median ROC gap 0.030 (second-best SELF 0.034) while keeping median AUC 0.787, versus tested-only 0.815 and y-obs 0.657.
- On the MIMIC-III sepsis task at q_t=1.5, k=4, DCEM has the highest median AUC among baselines (0.620 vs DragonNet 0.593), close to the oracle y-model (0.633).
- DCEM has the tightest ROC gap range on synthetic data (0.048) and worst-case ROC gap 0.060 vs 0.083 for the second-best baseline.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DCEM instead of the y-obs model | Synthetic classification · Synthetic, q_y = 0.5, k=1, q_t=2 | median AUC: 0.657 -> 0.787 | Helped |
| Use DCEM instead of SELF | Synthetic classification · Synthetic, q_y = 0.5, k=1, q_t=2 | median AUC: 0.677 vs. DCEM: 0.787 | Helped |
| Use DCEM instead of tested-only training | Synthetic classification · Synthetic, q_y = 0.5, k=1, q_t=2 | median AUC: DCEM: 0.787 vs. tested-only: 0.815 | Hurt |
| Use DCEM instead of SELF | Synthetic classification · Synthetic, q_y = 0.5, k=1, q_t=2 | median ROC gap: 0.034 -> 0.030 | Helped |
| Use DCEM instead of tested-only training | Synthetic classification across s_Y · Synthetic, q_y = 0.5, k=1, q_t=2 | maximum ROC gap: 0.083 -> 0.060 | Helped |
| Use DCEM instead of tested-only training | Synthetic classification across s_Y · Synthetic, q_y = 0.5, k=1, q_t=2 | minimum AUC: 0.623 -> 0.768 | Helped |
| Use DCEM instead of tested-only training | Synthetic classification at similar AUC · Synthetic, AUC in (0.825, 0.875) | average ROC gap: 0.050 -> 0.028 | Helped |
| Use DCEM instead of DragonNet | MIMIC-III sepsis classification · MIMIC-III sepsis, q_t=1.5, k=4 | median AUC: 0.593 -> 0.620 | Helped |
| Use DCEM instead of Group 0 | MIMIC-III sepsis classification · MIMIC-III sepsis, q_t=1.5, k=4 | median AUC: 0.342 vs. DCEM: 0.620 | Helped |
| Use DCEM instead of Group peer loss | MIMIC-III sepsis classification · MIMIC-III sepsis, q_t=1.5, k=4 | median ROC gap: Group peer loss ... 0.070 ... (DCEM: 0.105) | Hurt |
| Use DCEM instead of DragonNet | MIMIC-III sepsis classification across s_T · MIMIC-III sepsis, q_t=1.5, k=4 | minimum AUC: 0.574 -> 0.584 | Helped |
| Use DCEM instead of DragonNet | MIMIC-III sepsis classification across s_T · MIMIC-III sepsis, q_t=1.5, k=4 | AUC range: DCEM: 0.065 vs. DragonNet: 0.018 | Mixed |
Surprising findings
- The tested-only model, which discards untested individuals, is a strong baseline and achieves higher AUC than DCEM on synthetic data.
- Many approaches, including DCEM, achieve lower ROC gap than training on the true outcome y, showing that optimizing AUC on y does not guarantee bias mitigation.
Limitations
- EM can converge slowly since it is iterative.
- DCEM does not learn a full generative model over all variables.
- Evaluation relies on observed labels as ground truth, which is only approximately valid (about 90% of sepsis cohort tested), and synthetic experiments use low-dimensional simplified settings.
Experimental setup
Datasets: MIMIC-III Sepsis-3 cohort synthetic disparate censorship simulations
Benchmarks: synthetic disparate censorship simulations MIMIC-III sepsis classification
Metrics: AUC ROC gap (area between ROC curves) worst-case AUC worst-case ROC gap range
Baselines: y-obs model group a only tested-only group peer loss SELF DragonNet SAREM DivideMix truncated lq loss generalized Jensen-Shannon loss
Builds on: expectation-maximization DragonNet SELF peer loss SAREM
Closest papers by meaning
- Adaptive Labeling for Efficient Out-of-distribution Model Evaluation (NeurIPS 2024 · similarity 0.85)
- Navigating Towards Fairness with Data Selection (AAAI 2025 · similarity 0.85)
- Regretful Decisions under Label Noise (ICLR 2025 · similarity 0.84)
- Towards Fairness under Label Bias in Image Segmentation: Impact, Measurement and Mitigation (NeurIPS 2026 · similarity 0.84)
- Learning with Selectively Labeled Data from Multiple Decision-makers (ICML 2025 · similarity 0.84)
- Coherent Hierarchical Multi-Label Learning to Defer for Medical Imaging (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.