Research map / Large language models / Instruction and fine-tuning

Fine-Tuning Large Language Model Based Explainable Recommendation with Explainable Quality Reward

AAAI 2024 · Large language models › Instruction and fine-tuning · approach: Efficient LLM fine-tuning systems · 28 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes LLM2ER, a GPT-2-based explainable recommender with concept-graph-derived personalized prompts, and fine-tunes it with PPO using two unsupervised explainable quality rewards (BERT-based concept consistency via contrastive learning and GAN-style high-quality alignment) to yield LLM2ER-EQR.

Problem

LLM-based explainable recommendation generates explanations that lack personalization, are inconsistent with user preferences and item features, and are trained on questionable, noisy review data.

Approach

Combine a concept-graph (HGT) based rating predictor and beam-searched candidate concepts in the prompt with two unsupervised reward models, a contrastive concept-consistency reward (BERT similarity) and a GAN discriminator reward trained on filtered high-quality explanations, used to fine-tune the LLM via PPO.

Key results

• LLM2ER-EQR achieves the highest BLEU, ROUGE, Distinct, COR and CMR across all three datasets, with average CMR improvement of about 5.8% over the best baseline.
• On Amazon, LLM2ER-EQR reaches CMR 2.353 vs 2.146 for PRAG and Distinct-2 67.044 vs 64.420 for PRAG (approx. 4% gain).
• Ablations show LLM2ER-EQR outperforms LLM2ER backbone on all eight metrics, indicating the reward models improve explanation quality.

Measured effects reported in the paper

Surprising findings

• Concept-overlap-based diversity (COR) and concept-matching ratio trade off: adding more candidate concepts raises CMR but increases COR, reducing personalization.

Limitations

• Relies on pre-trained LLM, inheriting risks such as offensive language and social bias; post-processing needed for deployment.
• High-quality explanation set depends on an empirically set threshold delta and concept-overlap heuristic.
• Evaluated only on explanation text metrics with automatic measures; no human evaluation reported.

Experimental setup

Datasets: Amazon (Movie & TV) Yelp (2019) TripAdvisor

Benchmarks: Amazon (Movie & TV) Yelp (2019) TripAdvisor

Metrics: BLEU-1 BLEU-4 ROUGE-1 ROUGE-L Distinct-1 Distinct-2 COR CMR

Baselines: CAML NETE ReXPlug PETER PEVAE PRAG CVAEs

Builds on: InstructGPT PETER HGT PPO GPT-2

Closest papers by meaning

• Coherency Improved Explainable Recommendation via Large Language Model (AAAI 2025 · similarity 0.88)
• Re2LLM: Reflective Reinforcement Large Language Model for Session-based Recommendation (AAAI 2025 · similarity 0.88)
• LLMRG: Improving Recommendations through Large Language Model Reasoning Graphs (AAAI 2024 · similarity 0.88)
• R$^2$ec: Towards Large Recommender Models with Reasoning (NeurIPS 2025 · similarity 0.87)
• Reinforced Latent Reasoning for LLM-based Recommendation (ICLR 2026 · similarity 0.87)
• ConvRec-R1: Training LLM-based Conversational Recommender Systems with Reinforcement Learning (ICLR 2026 · similarity 0.87)