Research map / Probabilistic and scientific modeling / Treatment effects
Addressing Hidden Confounding with Heterogeneous Observational Datasets for Recommendation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MetaDebias, a meta-learning debiasing method that uses heterogeneous observational data (some with hidden confounding, some without) to correct selection bias in recommendation without RCT data.
Problem
Selection bias in recommender system ratings caused by hidden confounders that affect both observation and feedback; existing fixes need strong sensitivity assumptions or costly RCT data.
Approach
Use an identifiable propensity score π(x,g) and naive error imputation m(x,g) conditioned on a data-source indicator g, derive the oracle error imputation from them with a residual correction for hidden confounding, and train via bi-level meta-optimization (assumed updates) so the prediction model minimizes the imputed oracle error.
Key results
- On COAT, MetaDebias reaches AUC 0.746 and NDCG@5 0.473, above the best baseline Res-DR (0.740 AUC, 0.467 NDCG@5).
- On YAHOO! R3, MetaDebias reaches Recall@5 0.688 and NDCG@5 0.544 versus Res-DR's 0.678 and 0.538.
- On KuaiRec, MetaDebias achieves AUC 0.840 and NDCG@50 0.584 versus Res-DR's 0.838 and 0.580, and stays best across hidden confounding strengths and data proportions.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MetaDebias instead of Res-DR | Recommendation with heterogeneous observational data · COAT | AUC: 0.740±0.006 -> 0.746∗ ±0.007 | Helped |
| Use MetaDebias instead of AutoDebias | Recommendation with heterogeneous observational data · COAT | Recall@5: 0.501±0.012 -> 0.510∗ ±0.008 | Helped |
| Use MetaDebias instead of Res-DR | Recommendation with heterogeneous observational data · COAT | NDCG@5: 0.467±0.009 -> 0.473∗ ±0.010 | Helped |
| Use MetaDebias instead of Res-DR | Recommendation with heterogeneous observational data · YAHOO! R3 | AUC: 0.720±0.004 -> 0.722±0.004 | Helped |
| Use MetaDebias instead of Res-DR | Recommendation with heterogeneous observational data · YAHOO! R3 | Recall@5: 0.678±0.006 -> 0.688∗ ±0.005 | Helped |
| Use MetaDebias instead of Res-DR | Recommendation with heterogeneous observational data · YAHOO! R3 | NDCG@5: 0.538±0.006 -> 0.544∗ ±0.005 | Helped |
| Use MetaDebias instead of Res-DR | Recommendation with heterogeneous observational data · KUAIREC | AUC: 0.838±0.002 -> 0.840∗ ±0.001 | Helped |
| Use MetaDebias instead of Res-DR | Recommendation with heterogeneous observational data · KUAIREC | Recall@50: 0.852±0.004 -> 0.857∗ ±0.004 | Helped |
| Use MetaDebias instead of Res-DR | Recommendation with heterogeneous observational data · KUAIREC | NDCG@50: 0.580±0.003 -> 0.584∗ ±0.003 | Helped |
| Increase hidden-confounding strength by masking more features | Recommendation models on KUAIREC · KUAIREC | AUC | Hurt |
| Increase the share of data without hidden confounding | Recommendation models on KUAIREC · KUAIREC | AUC | Helped |
| Reduce training set size | Recommendation models on KUAIREC and YAHOO! R3 · KUAIREC and YAHOO! R3 | NDCG@K: 0.584 -> 0.576; 0.544 -> 0.534 | Hurt |
Surprising findings
- RCT-data-based methods (e.g., Bal, Res, LTD, AutoDebias) do not show significant improvements over RCT-free methods on the three datasets, which the authors attribute to MNAR data without hidden confounding being insufficient for effective calibration.
Limitations
- Assumes all confounders can be captured by sufficient feature collection; hidden confounders for partially observed samples may remain unobserved.
- Hidden confounding is simulated by feature masking on public datasets rather than real heterogeneous data.
- Bi-level optimization incurs longer training time than some baselines.
Experimental setup
Datasets: COAT YAHOO! R3 KuaiRec
Benchmarks: COAT YAHOO! R3 KuaiRec
Metrics: AUC Recall@K NDCG@K
Baselines: Naive IPS DR Stable-DR TDR ESMM Multi-IPS Multi-DR ESCM2-IPS ESCM2-DR BRD-IPS BRD-DR
Builds on: doubly robust estimator inverse propensity scoring BRD (bounded residual debiasing) Res-DR (residual model calibration with RCT data) AutoDebias
Closest papers by meaning
- Alleviating Dual Biases in Recommendation (Student Abstract) (AAAI 2025 · similarity 0.87)
- Relaxing the Accurate Imputation Assumption in Doubly Robust Learning for Debiased Collaborative Filtering (ICML 2024 · similarity 0.85)
- Tree of Preferences for Diversified Recommendation (NeurIPS 2025 · similarity 0.85)
- Fine-Grained Dynamic Framework for Bias-Variance Joint Optimization on Data Missing Not at Random (NeurIPS 2024 · similarity 0.85)
- Be Aware of the Neighborhood Effect: Modeling Selection Bias under Interference (ICLR 2024 · similarity 0.84)
- Debiased Collaborative Filtering with Kernel-Based Causal Balancing (ICLR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.