Research map / Multi-agent systems and games / Human-AI and goal-conditioned
Contrastive Representation for Interactive Recommendation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CRIR, a DDPG-based interactive recommender that adds a preference-ranking contrastive auxiliary loss (PRCL) over attention-weighted behavior representations to improve sample efficiency in cold-start settings.
Problem
Sample inefficiency of deep RL agents for interactive recommendation, where large action spaces and high-dimensional user state make training from scratch slow.
Approach
Refines the state representation with an auxiliary contrastive loss (Preference Ranking Contrastive Learning) that ranks interacted items by attention-based interest weights and uses positional weighted InfoNCE over behavior representations, avoiding computation over the full action set; trained in parallel with the DRL task via a mixed data sampling mechanism.
Key results
- On Virtual-Taobao under cold-start, CRIR reaches a good policy around episode 8000 while other methods have not reached that level within 20000 episodes.
- On the ML-1M-based simulator, CRIR outperforms the best baseline; most baselines except CRIR w/o CL and NICF fail within 2000 episodes.
- Mixed data sampling outperforms Divided and Combined sampling; constraining the critic loss with PRCL gives no improvement over the auxiliary training scheme.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add PRCL to the state representation network | CRIR with DDPG · Virtual-Taobao cold-start | cumulative reward | Helped |
| Add PRCL to the state representation network | CRIR with DDPG · ML-1M-oriented simulator cold-start | cumulative reward | Helped |
| Add the state representation network without PRCL (ablation) | DDPG interactive recommendation · Virtual-Taobao cold-start | cumulative reward | Helped |
| Add the state representation network without PRCL (ablation) | DDPG interactive recommendation · ML-1M-oriented simulator cold-start | cumulative reward | Helped |
| Increase PRCL frequency from 0.25 to 0.5 (ablation) | CRIR on Virtual-Taobao · Virtual-Taobao | episode reward | Helped |
| Increase PRCL frequency from 0.5 to 0.75 (ablation) | CRIR on Virtual-Taobao · Virtual-Taobao | episode reward | Helped |
| Replace discriminative positional coefficients with a balanced coefficient (ablation) | PRCL on Virtual-Taobao · Virtual-Taobao | episode reward | Hurt |
| Use the Mixed Mechanism instead of Divided or Combined sampling (ablation) | CRIR with PRCL · Virtual-Taobao | performance | Helped |
| Use Constrained Mechanism instead of Auxiliary Mechanism (ablation) | CRIR with PRCL · Virtual-Taobao | performance | Hurt |
| Update the encoding layer with both RL and PRCL gradients rather than PRCL gradients only (ablation) | CRIR representation encoder · Virtual-Taobao | episode reward | Hurt |
Surprising findings
- Using only the PRCL gradient to update the encoder performs nearly as well as, and at the end exceeds, the combined RL+PRCL update, suggesting conflict between RL and contrastive gradients.
- Adding PRCL as a constraint to the critic loss yields no improvement over the auxiliary training mechanism.
Limitations
- Evaluated only on simulators (Virtual-Taobao and a self-constructed ML-1M simulator), not on real user traffic.
- Baselines are limited to five methods, chosen partly because other sample-efficiency IR works were deemed inapplicable.
- Only results from a single backbone (DDPG) are reported; the time complexity claim and some method details are loosely specified.
Experimental setup
Datasets: Virtual-Taobao MovieLens-1M
Benchmarks: Virtual-Taobao ML-1M-based simulator
Metrics: cumulative reward click-through rate (CTR)
Baselines: SAC CRR PPO DRR NICF
Builds on: DDPG Deep Interest Network Priority Experience Replay DRR InfoNCE
Closest papers by meaning
- VRAgent-R1: Boosting Video Recommendation with MLLM-based Agents via Reinforcement Learning (AAAI 2026 · similarity 0.86)
- SRA-CL: Semantic Retrieval Augmented Contrastive Learning for Sequential Recommendation (NeurIPS 2025 · similarity 0.86)
- On the Unexpected Effectiveness of Reinforcement Learning for Sequential Recommendation (ICML 2024 · similarity 0.86)
- Review-Enhanced Hierarchical Contrastive Learning for Recommendation (AAAI 2024 · similarity 0.85)
- ConvRec-R1: Training LLM-based Conversational Recommender Systems with Reinforcement Learning (ICLR 2026 · similarity 0.85)
- The Adaptive Q-Network for Recommendation Tasks with Dynamic Item Space (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.