Research map / Large language models / Retrieval and knowledge
LLMRG: Improving Recommendations through Large Language Model Reasoning Graphs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LLMRG, which prompts GPT-3.5/GPT-4 to build personalized reasoning graphs from user profiles and behavior sequences, encodes them with SR-GNN, and fuses them with sequential recommenders to improve HR and NDCG on three datasets.
Problem
Sequential recommenders lack interpretability and fail to capture higher-level causal and logical relationships between user behaviors and profiles.
Approach
Use an LLM to build personalized reasoning graphs via chained graph reasoning, divergent extension, and abductive self-verification with masking and scoring, cache validated chains in a knowledge base, and encode the graphs with SR-GNN as extra input to a base recommender (DuoRec).
Key results
- LLMRG (GPT-4) on top of DuoRec improves HR@5 on ML-1M by 14.76% and NDCG@5 by 26.01% relative to DuoRec (0.2011 baseline HR@5).
- LLMRG (GPT-4) improves Amazon Beauty HR@5 by 11.93% over DuoRec, and GPT-4 consistently outperforms GPT-3.5 variants.
- Knowledge base self-improvement reduces LLM call frequency by about 30% after 3,000 reasoning and verification steps.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add LLM reasoning graphs to DuoRec | DuoRec · ML-1M | ND@5: + 26.01 % | Helped |
| Add LLM reasoning graphs to DuoRec | DuoRec · Amazon Beauty | HR@5: + 11.93 % | Helped |
| Add LLM reasoning graphs to DuoRec | DuoRec · Amazon Clothing | HR@5: + 11.40 % | Helped |
| Use direct GPT3.5 recommendations without constructing a reasoning graph (ablation) | DuoRec · Amazon Beauty | HR@5: - 1.26 % | Hurt |
| Add a sequence graph without reasoning (ablation) | DuoRec · ML-1M | HR@5: + 6.36 % | Helped |
| Remove divergent extension (ablation) | LLMRG (GPT3.5) · ML-1M | HR@5: + 5.12 % | Helped |
| Add divergent extension (ablation) | LLMRG (GPT3.5) · ML-1M | HR@5: + 12.87 % | Helped |
| Remove self-verification (ablation) | LLMRG (GPT3.5) · ML-1M | HR@5: - 4.72 % | Hurt |
| Add self-verification (ablation) | LLMRG (GPT3.5) · ML-1M | HR@5: + 12.87 % | Helped |
| Remove divergent extension (ablation) | LLMRG (GPT4) · ML-1M | HR@5: + 7.06 % | Helped |
| Add divergent extension (ablation) | LLMRG (GPT4) · ML-1M | HR@5: + 14.76 % | Helped |
| Remove self-verification (ablation) | LLMRG (GPT4) · ML-1M | HR@5: + 5.86 % | Hurt |
Surprising findings
- Naive DuoRec+GPT-3.5 (LLM recommending directly without a reasoning graph) slightly decreases performance on Amazon Beauty.
- Removing self-verification from LLMRG (GPT-3.5) lowers performance below DuoRec-level gains on ML-1M, showing uncontrolled reasoning adds noise.
Limitations
- Gains depend on LLM capability; GPT-3.5 variant gives smaller and sometimes negative effects.
- Performance is sensitive to verification threshold tau; too high a threshold sparsifies the graph and hurts results on Amazon data.
- Generated items may not exist in the item catalog and need a separate small language model for matching.
Experimental setup
Datasets: ML-1M Amazon Beauty Amazon Clothing
Benchmarks: ML-1M Amazon Beauty Amazon Clothing
Metrics: HR@5 HR@10 NDCG@5 NDCG@10
Baselines: FDSA BERT4Rec CL4SRec DuoRec DuoRec w/ seq DuoRec+GPT3.5 DuoRec+GPT4
Builds on: DuoRec SR-GNN chain-of-thought prompting
Closest papers by meaning
- Bridging the User-side Knowledge Gap in Knowledge-aware Recommendations with Large Language Models (AAAI 2025 · similarity 0.90)
- Can LLMs Outshine Conventional Recommenders? A Comparative Evaluation (NeurIPS 2025 · similarity 0.89)
- Improving LLMs for Recommendation with Out-Of-Vocabulary Tokens (ICML 2025 · similarity 0.89)
- R$^2$ec: Towards Large Recommender Models with Reasoning (NeurIPS 2025 · similarity 0.88)
- Reinforced Latent Reasoning for LLM-based Recommendation (ICLR 2026 · similarity 0.88)
- CoT4Rec: Revealing User Preferences Through Chain of Thought for Recommender Systems (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.