Research map / Large language models / Agents, code and math
LLM-Powered User Simulator for Recommender System
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes an LLM-powered user simulator for RL-based recommenders that uses ChatGLM-6B to distill item pros/cons keywords and ensembles two keyword/similarity logical models with a SASRec statistical model to output like/dislike.
Problem
Existing user simulators for RL-based recommender systems lack explicit modeling of user preferences and have no effective way to evaluate simulation fidelity, while direct LLM-based simulation is costly and prone to hallucination.
Approach
Use an LLM offline to extract keyword-level pros/cons for items (from item metadata and user comments), then combine rule-based keyword matching and embedding similarity models with a SASRec statistical model via majority-vote ensemble to simulate binary like/dislike feedback as the RL reward.
Key results
- On Yelp, the simulator reaches AUC 0.674 with 27.56 average reward and 330.98 total reward under DQN, versus SUBER (AUC 0.643) and KuaiSim (AUC 0.658).
- DQN achieves the highest average reward on four of five datasets (e.g., 16.70 on Amazon Music, 38.60 on Amazon Movie), while A2C leads on Amazon Games and Anime.
- Inference time is 0.76 s for the proposed simulator, versus 2.42 s for SUBER and 0.53 s for KuaiSim.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the proposed LLM-based logical and statistical simulator instead of SUBER | RL-based recommender evaluation on Yelp · Yelp | average reward: 23.74 -> 27.56 | Helped |
| Use the proposed LLM-based logical and statistical simulator instead of KuaiSim | RL-based recommender evaluation on Yelp · Yelp | average reward: 25.35 -> 27.56 | Helped |
| Use the proposed LLM-based logical and statistical simulator instead of SUBER | RL-based recommender evaluation on Yelp · Yelp | total reward: 297.48 -> 330.98 | Helped |
| Use the proposed LLM-based logical and statistical simulator instead of KuaiSim | RL-based recommender evaluation on Yelp · Yelp | total reward: 316.46 -> 330.98 | Helped |
| Use the proposed LLM-based logical and statistical simulator instead of SUBER | RL-based recommender evaluation on Yelp · Yelp | AUC: 0.643 -> 0.674 | Helped |
| Use the proposed LLM-based logical and statistical simulator instead of KuaiSim | RL-based recommender evaluation on Yelp · Yelp | AUC: 0.658 -> 0.674 | Helped |
| Use the proposed simulator instead of SUBER or KuaiSim | User simulator · Yelp | inference time (s): 2.42, 0.53 -> 0.76 | Mixed |
Surprising findings
- The LLM is used only offline for keyword extraction, so the simulator avoids LLM calls during RL training, which the authors argue sidesteps LLM cost and hallucination during training.
Limitations
- Simulator only predicts binary like/dislike, not ratings, duration, or retention.
- Simulation accuracy evaluated mainly through RL reward and AUC rather than a direct fidelity benchmark against real user feedback.
- Similarity model and keyword matching may be less precise for items with new genres, as the authors note in the case study.
Experimental setup
Datasets: Yelp Amazon Music Amazon Games Amazon Movie Anime
Benchmarks: Yelp Amazon Music Amazon Games Amazon Movie Anime
Metrics: average reward total reward liking ratio AUC inference time
Baselines: SUBER KuaiSim RecoGym RecSim VirtualTaobao Agent4Rec
Builds on: SASRec BERT Chain-of-Thought
Closest papers by meaning
- Would You Like Your Data to Be Trained? A User Controllable Recommendation Framework (AAAI 2024 · similarity 0.88)
- ConvRec-R1: Training LLM-based Conversational Recommender Systems with Reinforcement Learning (ICLR 2026 · similarity 0.88)
- Objective Shaping with Hard Negatives: Windowed Partial AUC Optimization for RL-based LLM Recommenders (NeurIPS 2026 · similarity 0.87)
- Preference Diffusion for Recommendation (ICLR 2025 · similarity 0.86)
- Re2LLM: Reflective Reinforcement Large Language Model for Session-based Recommendation (AAAI 2025 · similarity 0.86)
- On the Unexpected Effectiveness of Reinforcement Learning for Sequential Recommendation (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.