Research map / Large language models / Retrieval and knowledge
Re2LLM: Reflective Reinforcement Large Language Model for Session-based Recommendation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Re2LLM keeps a GPT-4 backbone frozen, has it self-reflect on recommendation errors to build a hint knowledge base, and trains a lightweight PPO retrieval agent to select hints for session-based recommendation.
Problem
Session-based recommendation with LLMs: prompt-based methods lack task-specific feedback for finding good prompts, while fine-tuning is computationally expensive and tied to open-source backbones.
Approach
Use the LLM's own self-reflection to generate and filter a non-redundant knowledge base of natural-language hints from incorrect predictions, then learn a lightweight PPO retrieval agent (state from BERT embeddings of session) that picks which hint to add to the prompt using reward equal to the NDCG improvement from the hint-enhanced prompt, keeping the LLM frozen.
Key results
- In the few-shot setting, Re2LLM reaches HR@10 0.5735 on Movie (vs 0.5110 best baseline LLMRank) and 0.7173 on Game (vs 0.6636), with significant gains reported.
- Re2LLM with 500 few-shot samples outperforms full-data deep baselines on Movie HR@10 (0.5735 vs 0.5895 GCEGNN is not exceeded, so comparable rather than uniformly better) and on Game HR@10 (0.7173 vs 0.5838 AttenMixer).
- Ablation: removing the DRL retrieval agent and using random hints drops Movie NDCG@10 from 0.3358 to 0.2908.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Re2LLM instead of the strongest baseline | Movie few-shot · Movie few-shot | HR@5: 0.3649 -> 0.4126 | Helped |
| Use Re2LLM instead of the strongest baseline | Movie few-shot · Movie few-shot | HR@10: 0.5110 -> 0.5735 | Helped |
| Use Re2LLM instead of the strongest baseline | Movie few-shot · Movie few-shot | NDCG@5: 0.2436 -> 0.2839 | Helped |
| Use Re2LLM instead of the strongest baseline | Movie few-shot · Movie few-shot | NDCG@10: 0.2795 -> 0.3358 | Helped |
| Use Re2LLM instead of the strongest baseline | Game few-shot · Game few-shot | HR@5: 0.5291 -> 0.5664 | Helped |
| Use Re2LLM instead of the strongest baseline | Game few-shot · Game few-shot | HR@10: 0.6636 -> 0.7173 | Helped |
| Use Re2LLM instead of the strongest baseline | Game few-shot · Game few-shot | NDCG@5: 0.4016 -> 0.4195 | Helped |
| Use Re2LLM instead of the strongest baseline | Game few-shot · Game few-shot | NDCG@10: 0.4531 -> 0.4689 | Helped |
| Add hint-enhanced prompts (ablation) | Re2LLM · Movie | NDCG@10: 0.3198 -> 0.3358 | Helped |
| Add hint-enhanced prompts (ablation) | Re2LLM · Game | NDCG@10: 0.4550 -> 0.4689 | Helped |
| Use automated filtering of hints (ablation) | Re2LLM · Movie | NDCG@10: 0.3289 -> 0.3358 | Helped |
| Use automated filtering of hints (ablation) | Re2LLM · Game | NDCG@10: 0.4602 -> 0.4689 | Helped |
Surprising findings
- Randomly selected hints can hurt LLM recommendation performance, sometimes below the no-hint baseline.
- Using too few retrieval-agent training samples (under 50) yields performance even below random hint selection.
Limitations
- Relies on closed-source GPT-4 API with per-sample costs
- Evaluation uses a 2,000-session test subset with 49 sampled negatives
- Only single-hint selection per session in the current design
Experimental setup
Datasets: Hetrec2011-MovieLens Amazon Video Game
Benchmarks: Hetrec2011-MovieLens Amazon Game
Metrics: HR@5 HR@10 NDCG@5 NDCG@10
Baselines: FPMC GRU4Rec NARM SRGNN GCEGNN AttenMixer LLMRank NIR
Builds on: PPO Chain-of-Thought Reflexion LLMRank NIR
Closest papers by meaning
- ConvRec-R1: Training LLM-based Conversational Recommender Systems with Reinforcement Learning (ICLR 2026 · similarity 0.90)
- Active Large Language Model-Based Knowledge Distillation for Session-Based Recommendation (AAAI 2025 · similarity 0.89)
- R$^2$ec: Towards Large Recommender Models with Reasoning (NeurIPS 2025 · similarity 0.89)
- Reinforced Latent Reasoning for LLM-based Recommendation (ICLR 2026 · similarity 0.89)
- SLMRec: Distilling Large Language Models into Small for Sequential Recommendation (ICLR 2025 · similarity 0.89)
- On the Unexpected Effectiveness of Reinforcement Learning for Sequential Recommendation (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.