Research map / Probabilistic and scientific modeling / Causal discovery
Causal Representation Learning for Generalisable Recommendation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes an information-theoretic causal disentanglement objective (utility CMI minus confounder MI penalty), made tractable via InfoNCE and NCE-CLUB bounds, and shows gains under distribution shift on a synthetic SCM, KuaiRand, and a Spotify playlist A/B test.
Problem
Recommender models trained on confounded, policy-generated interaction logs fail to generalise to the candidate distribution at serving time, so offline metrics poorly predict online performance.
Approach
Learn a treatment representation that maximises conditional mutual information with the outcome given the observed confounder while penalising its mutual information with the confounder; the penalty is estimated with a gradient-reversal adversary using an InfoNCE lower bound (monitored with NCE-CLUB), giving a deployable add-on to any supervised ranker.
Key results
- On KuaiRand, CRL (lambda=0.3) improves random-exposure test AUC from 0.6453 to 0.6582 (+0.0129) over 500 seeds while biased validation AUC is slightly lower (0.7508 vs 0.7574).
- In the Spotify online A/B test over two weeks, the CRL variant increased track streams by +0.75% and minutes played by +0.50% and reduced track skips by -0.61%, all significant at 95%.
- On the synthetic SCM, the CRL variant's intervention sensitivity stays near zero across outcome-noise levels while the baseline CATE model is sensitive to non-causal interventions, with matched MAE.
Surprising findings
- Offline in-distribution metrics (biased validation AUC, synthetic MAE, production offline metrics) were essentially unchanged, so gains only appear under shifted evaluation.
- The Spotify A/B gains are described as the same order of magnitude as the improvement the production ranker gave over its mature predecessor.
Limitations
- Assumptions A1 and A2 cannot be empirically verified on real data.
- Online evaluation covers a single product surface (Spotify playlist generation).
- Critic may need more expressive capacity under richer confounding structures.
Experimental setup
Datasets: KuaiRand-Pure synthetic linear SCM Spotify production interaction logs
Benchmarks: KuaiRand-Pure random-exposure test split synthetic SCM intervention sensitivity Spotify online A/B test
Metrics: AUC Recall@5 NDCG@5 MAE intervention sensitivity track streams track skips minutes played
Baselines: DeepFM baseline (production ranker) CFR/TARNet CEVAE Dragonnet IPS reweighting
Builds on: InfoNCE CLUB DeepFM gradient reversal invariant risk minimisation
Closest papers by meaning
- Learning Multiple User Distributions for Recommendation via Guided Conditional Diffusion (AAAI 2025 · similarity 0.86)
- Preference Diffusion for Recommendation (ICLR 2025 · similarity 0.85)
- Alleviating Dual Biases in Recommendation (Student Abstract) (AAAI 2025 · similarity 0.85)
- Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations (ICML 2024 · similarity 0.84)
- Influential Exemplar Replay for Incremental Learning in Recommender Systems (AAAI 2024 · similarity 0.84)
- On Efficiency-Effectiveness Trade-off of Diffusion-based Recommenders (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.