Research map / Label-efficient and robust learning / Test-time adaptation
Multiply Robust Estimation for Local Distribution Shifts with Multiple Domains
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a two-stage "multiply robust" estimator for tabular supervised learning under local (per-segment) covariate or label shift, combining clustered base models with segment-specific linear stacking and an XGBoost refinement step.
Problem
Predicting responses under distribution shift when shifts are only assumed to be covariate or label shift locally within each segment of a population with multiple segments, and training data per segment may be scarce.
Approach
Cluster segments by MMD distance and train base models per cluster to share information; learn a per-segment linear combination of base models (stage 1) and then refine with importance-weighted, regularized stage-2 model penalized toward the stage-1 prior (stage 2), with a generalization bound.
Key results
- On Apartment regression, MR achieves relative MSE 0.524 versus XGB 1.0 (lower is better), the best among compared methods.
- On the Meta user city dataset, MR reaches relative CE 0.852 and Brier 0.864 versus the production model, outperforming DR (0.93) and DR-SF (0.917).
- MR is best on all five UCI datasets (e.g., Adult 0.901, Banking 0.925, Customer 0.834) while MFSAN and DADIL perform worse than XGB on most tasks.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use multiply robust estimation instead of DR-SF | XGB-based segment adaptation · Adult | cross entropy: 0.948 (0.027) -> 0.901 (0.028) | Helped |
| Use multiply robust estimation instead of DR-SF | XGB-based segment adaptation · Banking | cross entropy: 0.971 (0.03) -> 0.925 (0.03) | Helped |
| Use multiply robust estimation instead of MFSAN | Multi-source domain adaptation · Customer | cross entropy: 0.853 (0.008) -> 0.834 (0.008) | Helped |
| Use multiply robust estimation instead of DR-SF | XGB-based segment adaptation · Apartment | MSE: 0.774 (0.01) -> 0.524 (0.008) | Helped |
| Use multiply robust estimation instead of DR-SF | XGB-based segment adaptation · Online News | MSE: 0.994 (0.023) -> 0.949 (0.023) | Helped |
| Use multiply robust estimation instead of DR-SF | XGB-based segment adaptation · Meta user city prediction | cross entropy: 0.917 (0.011) -> 0.852 (0.012) | Helped |
| Use multiply robust estimation instead of DR-SF | XGB-based segment adaptation · Meta user city prediction | Brier score: 0.927 (0.013) -> 0.864 (0.013) | Helped |
| Add one-hot segment features to DR | DR · Adult, Banking, Customer, Apartment, and Online News | test loss: 0.953 (0.027) -> 0.948 (0.027); 0.976 (0.03) -> 0.971 (0.03); 0.978 (0.009) -> 1.025 (0.01); 1.0 (0.013) -> 0.774 (0.01) | Mixed |
Surprising findings
- MFSAN and DADIL, multi-source domain adaptation methods, underperform the plain XGB baseline on most datasets, and are far slower (around 47-60 minutes versus 5 minutes for MR on the Meta data).
Limitations
- Theory relies on restrictive assumptions (RKHS function class, bounded importance weights, sub-Gaussian conditions), which the authors say are not necessary in practice.
- Distance matrix computation for segment clustering is O(n_tr^2), the most expensive step.
- Experiments use semi-synthetic shifts constructed on UCI data; the Meta dataset is proprietary.
Experimental setup
Datasets: Adult Banking Customer Apartment Online News Meta user city dataset
Benchmarks: Adult Banking Customer Apartment Online News Meta user city dataset simulated covariate shift regression
Metrics: MSE cross entropy Brier score
Baselines: XGB DR DR-SF MFSAN DADIL
Builds on: doubly robust estimation for covariate shift black box shift estimation kernel mean matching XGBoost
Closest papers by meaning
- Distributionally Robust Models with Parametric Likelihood Ratios (ICLR 2022 · similarity 0.87)
- Robust Generalization despite Distribution Shift via Minimum Discriminating Information (NeurIPS 2021 · similarity 0.87)
- Bridging Multicalibration and Out-of-distribution Generalization Beyond Covariate Shift (NeurIPS 2024 · similarity 0.86)
- Optimal Aggregation of Prediction Intervals under Unsupervised Domain Shift (NeurIPS 2024 · similarity 0.85)
- Tolerant Algorithms for Learning with Arbitrary Covariate Shift (NeurIPS 2024 · similarity 0.85)
- Towards Robustness to Natural Variations and Distribution Shift (Student Abstract) (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.