Research map / Learning theory and algorithms / Kernels and regression theory
Bridging Multicalibration and Out-of-distribution Generalization Beyond Covariate Shift
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Extends multicalibration to joint covariate-label grouping functions, shows equivalence with invariance under concept shift, and proposes MC-Pseudolabel, a hyperparameter-free post-processing regression algorithm for OOD regression.
Problem
Out-of-distribution generalization for regression under both covariate shift and concept shift (Y|X changes), where model selection is hard without target validation data.
Approach
Extend multicalibration to grouping functions defined on covariates and outcomes jointly (density ratios p_e(x,y)/p_S(x,y)); show multicalibration w.r.t. these is equivalent to approximate invariance; propose MC-Pseudolabel, which iteratively regresses pseudolabels on level sets of the model using grouping functions, without extra hyperparameters.
Key results
- On ACSIncome with ID validation, MC-Pseudolabel achieves RMSE 0.428 vs 0.457 for the best invariance baseline MIP and 0.455 for CLOvE.
- On PovertyMap with ID validation, MC-Pseudolabel reaches worst-U/R Pearson 0.50 vs 0.47 for MRI.
- On VesselPower with ID validation, MC-Pseudolabel achieves RMSE 1.61 vs 1.69 for CVaR and chi^2-DRO, with comparable oracle-validation performance.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MC-Pseudolabel instead of the strongest baseline | ACSIncome · ID validation | RMSE: 0.455 -> 0.428 | Helped |
| Use MC-Pseudolabel instead of the strongest baseline | ACSIncome · Worst validation | RMSE: 0.449 -> 0.425 | Helped |
| Use MC-Pseudolabel instead of the strongest baseline | ACSIncome · Oracle validation | RMSE: 0.433 -> 0.411 | Helped |
| Use MC-Pseudolabel instead of the strongest baseline | PovertyMap · ID validation | Worst-U/R Pearson: 0.47 -> 0.50 | Helped |
| Use MC-Pseudolabel instead of the strongest baseline | PovertyMap · Worst validation | Worst-U/R Pearson: 0.46 -> 0.50 | Helped |
| Use MC-Pseudolabel instead of the strongest baseline | PovertyMap · Oracle validation | Worst-U/R Pearson: 0.51 vs 0.51 | No clear effect |
| Use MC-Pseudolabel instead of the strongest baseline | VesselPower · ID validation | RMSE: 1.69 -> 1.61 | Helped |
| Use MC-Pseudolabel instead of the strongest baseline | VesselPower · Oracle validation | RMSE: 1.49 -> 1.52 | Hurt |
Surprising findings
- Joint grouping functions can make a perfectly multicalibrated predictor impossible to exist (e.g., h(x,y)=y), so the algorithm outputs the initial model in that case.
- MC-Pseudolabel can output a predictor with higher source risk than its input, because the projection step breaks risk monotonicity.
Limitations
- Current algorithm targets regression; extension to classification is left open.
- Convergence guarantee proven only for multivariate normal data and in the representation-learning setting without rounding.
- Joint grouping function class is not closed under affine transformation, so perfect multicalibration may not exist.
Experimental setup
Datasets: PovertyMap (WILDS) ACSIncome VesselPower (Shifts) synthetic multi-environment linear Gaussian data
Benchmarks: PovertyMap ACSIncome VesselPower
Metrics: Worst-U/R Pearson RMSE
Baselines: ERM IRM MIP IB-IRM CLOvE MRI REX Fishr IDGM GroupDRO CVaR JTT
Builds on: multicalibration Invariant Risk Minimization LSBoost DomainBed JTT
Closest papers by meaning
- Unified Approach for Weakly Supervised Multicalibration (NeurIPS 2026 · similarity 0.87)
- Discretization-free Multicalibration through Loss Minimization over Tree Ensembles (NeurIPS 2025 · similarity 0.86)
- Multiply Robust Estimation for Local Distribution Shifts with Multiple Domains (ICML 2024 · similarity 0.86)
- Sufficient Invariant Learning for Distribution Shift (CVPR 2025 · similarity 0.86)
- LaSCal: Label-Shift Calibration without target labels (NeurIPS 2024 · similarity 0.85)
- Not all distributional shifts are equal: Fine-grained robust conformal inference (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.