Research map / Large language models / Language, safety and interpretability
LLM-ESR: Large Language Models Enhancement for Long-tailed Sequential Recommendation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LLM-ESR, which caches LLM (text-embedding-ada-002) item and user embeddings to enhance sequential recommenders via dual-view modeling and retrieval-augmented self-distillation for long-tail users and items.
Problem
Sequential recommendation suffers from long-tail users (few interactions) and long-tail items (rarely consumed), where collaborative-only methods perform poorly and existing LLM integrations either add inference cost or erode semantics.
Approach
Freezes cached LLM embeddings of items and users, combines a frozen-semantic view with a collaborative view through a shared sequence encoder and two-level fusion, and transfers knowledge from semantically retrieved similar users via self-distillation, so no LLM is called at inference.
Key results
- On Yelp with SASRec backbone, LLM-ESR reaches overall H@10 0.6673 and N@10 0.4208, versus 0.6415 / 0.3997 for the best baseline LLMInit.
- On Yelp with Bert4Rec, tail-item H@10 rises to 0.1227 from 0.0874 for LLMInit.
- On Amazon Beauty with SASRec, overall H@10 is 0.5672 versus 0.5455 for LLMInit, with the tail-item H@10 at 0.2257 versus 0.1714.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply LLM-ESR | GRU4Rec · Yelp | H@10: 0.4985 -> 0.5724 | Helped |
| apply LLM-ESR | Bert4Rec · Fashion | H@10: 0.4926 -> 0.5487 | Helped |
| apply LLM-ESR | SASRec · Beauty | H@10: 0.5455 -> 0.5672 | Helped |
| add collaborative view (ablation) | LLM-ESR with SASRec · Yelp overall | H@10: 0.6320 -> 0.6673 | Helped |
| add semantic view (ablation) | LLM-ESR with SASRec · Yelp tail items | H@10: 0.1105 -> 0.1893 | Helped |
| add retrieval-augmented self-distillation (ablation) | LLM-ESR with SASRec · Yelp tail users | H@10: 0.6566 -> 0.6685 | Helped |
| share the sequence encoder across views (ablation) | LLM-ESR with SASRec · Yelp overall | H@10: 0.6595 -> 0.6673 | Helped |
| add cross-attention (ablation) | LLM-ESR with SASRec · Yelp overall | H@10: 0.6644 -> 0.6673 | Helped |
| use a two-layer adapter instead of a one-layer adapter (ablation) | LLM-ESR with SASRec · Yelp overall | H@10: 0.6108 -> 0.6673 | Helped |
| initialize collaborative embeddings from dimension-reduced semantic embeddings instead of random initialization (ablation) | LLM-ESR with SASRec · Yelp overall | H@10: 0.6440 -> 0.6673 | Helped |
| freeze the semantic embedding layer (ablation) | LLM-ESR with SASRec · Yelp, full prompts | H@10: 0.6069 -> 0.6673 | Helped |
| freeze the semantic embedding layer (ablation) | LLM-ESR with SASRec · Yelp, cropped prompts | H@10: 0.6025 -> 0.6477 | Helped |
Surprising findings
- Freezing the semantic embedding layer matters most with limited textual data; training it instead causes overfitting and lowers tail-item performance.
- CITIES sometimes falls below its backbone SRS model overall due to a seesaw effect, with drastic drops for popular items.
Limitations
- Two hyperparameters (alpha and N) are time-consuming to tune.
- Only OpenAI text-embedding-ada-002 embeddings were validated; other LLM embedders were not tested.
- Cold-start users (fewer than three interactions) are excluded from the study.
Experimental setup
Datasets: Yelp Amazon Fashion Amazon Beauty
Benchmarks: Yelp Amazon Fashion Amazon Beauty
Metrics: H@10 N@10
Baselines: CITIES MELT RLMRec LLMInit GRU4Rec Bert4Rec SASRec
Builds on: SASRec GRU4Rec Bert4Rec self-distillation
Closest papers by meaning
- LLMEmb: Large Language Model Can Be a Good Embedding Generator for Sequential Recommendation (AAAI 2025 · similarity 0.93)
- SLMRec: Distilling Large Language Models into Small for Sequential Recommendation (ICLR 2025 · similarity 0.93)
- LLM4RSR: Large Language Models as Data Correctors for Robust Sequential Recommendation (AAAI 2025 · similarity 0.93)
- Can LLMs Outshine Conventional Recommenders? A Comparative Evaluation (NeurIPS 2025 · similarity 0.91)
- Improving LLMs for Recommendation with Out-Of-Vocabulary Tokens (ICML 2025 · similarity 0.90)
- Language Ranker: A Lightweight Ranking framework for LLM Decoding (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.