Research map / Large language models / Reasoning and chain of thought
Think before Recommendation: Autonomous Reasoning-enhanced Recommender
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RecZero, a pure-RL (GRPO) method that trains a single Qwen2.5-7B-Instruct LLM to produce structured think-before-recommendation reasoning for rating prediction, and RecOne, an SFT cold-start plus RL variant.
Problem
Rating prediction with LLM reasoning currently relies on distillation from teacher LLMs via multi-model, multi-stage SFT, which suffers from weak teacher recommendation ability, costly static supervision, and superficial reasoning transfer.
Approach
Train a single LLM with GRPO using a structured reasoning template (analyze user, analyze item, match, rate) and rule-based rewards combining a format reward and a continuous MAE-style answer reward; RecOne adds a DeepSeek-R1-generated cold-start SFT before RL.
Key results
- RecOne reaches RMSE/MAE of 0.7784/0.5017 on Book, 0.6776/0.3816 on Music and 0.9774/0.7012 on Yelp, ranking first on all six metrics.
- RecZero beats all baselines on MAE on the three datasets (e.g., 0.4271 on Music vs 0.5352 for Reason4Rec), but not on RMSE on Book and Yelp.
- Ablation: correctness-only reward and SFT-only warm-start both underperform the full RecOne reward/RL pipeline.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use RecOne instead of Reason4Rec | Rating prediction · Book | RMSE: 0.8325 -> 0.7784 | Helped |
| Use RecOne instead of Reason4Rec | Rating prediction · Book | MAE: 0.5937 -> 0.5017 | Helped |
| Use RecOne instead of Reason4Rec | Rating prediction · Music | RMSE: 0.7647 -> 0.6776 | Helped |
| Use RecOne instead of Reason4Rec | Rating prediction · Music | MAE: 0.5352 -> 0.3816 | Helped |
| Use RecOne instead of Reason4Rec | Rating prediction · Yelp | RMSE: 0.9972 -> 0.9774 | Helped |
| Use RecOne instead of Reason4Rec | Rating prediction · Yelp | MAE: 0.7473 -> 0.7012 | Helped |
| Add cold-start SFT initialization before RL | RecZero · Book | MAE | Helped |
| Remove the explicit thinking process (ablation) | RecOne · Four datasets | MAE | Hurt |
| Replace multi-step user/item/match reasoning with single-step thinking (ablation) | RecOne · Four datasets | MAE | Hurt |
| Use correctness-only reward instead of the rating-distance reward (ablation) | RecOne · Four datasets | MAE | Hurt |
| Remove RL after warm-start and use SFT only (ablation) | RecOne · Four datasets | MAE | Hurt |
| Use pure RL rather than pure SFT | RecZero versus RecOne(SFT) · Amazon-Music | MAE: 0.6472 -> 0.5419 | Helped |
Surprising findings
- RecZero's MAE initially increased during the first ~20 RL steps, attributed to the model first learning format rewards.
- Correctness-only reward (integer exact-match) hurts MAE/RMSE relative to the continuous reward, since it discourages decimal predictions near true ratings.
Limitations
- Only a 7B base model was tested due to computational constraints, so larger-model RL potential is unassessed.
- Did not test whether RecZero/RecOne can replace teacher models for generating cold-start data in iterative self-improvement.
- RecZero does not beat the strongest baseline on RMSE for Book and Yelp.
Experimental setup
Datasets: Amazon-book Amazon-music Yelp
Benchmarks: Amazon-book Amazon-music Yelp
Metrics: MAE RMSE
Baselines: MF DeepCoNN NARRE DAML Rec-SAVER EXP3RT Reason4Rec RecOne(SFT)
Builds on: GRPO DeepSeek-R1-Zero Reason4Rec EXP3RT Rec-SAVER
Closest papers by meaning
- R$^2$ec: Towards Large Recommender Models with Reasoning (NeurIPS 2025 · similarity 0.90)
- Reinforced Latent Reasoning for LLM-based Recommendation (ICLR 2026 · similarity 0.89)
- Re2LLM: Reflective Reinforcement Large Language Model for Session-based Recommendation (AAAI 2025 · similarity 0.88)
- Objective Shaping with Hard Negatives: Windowed Partial AUC Optimization for RL-based LLM Recommenders (NeurIPS 2026 · similarity 0.87)
- On the Unexpected Effectiveness of Reinforcement Learning for Sequential Recommendation (ICML 2024 · similarity 0.87)
- ConvRec-R1: Training LLM-based Conversational Recommender Systems with Reinforcement Learning (ICLR 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.