Research map / Large language models / Retrieval and knowledge

LEARN: Knowledge Adaptation from Large Language Model to Recommendation for Practical Industrial Application

AAAI 2025 · Large language models › Retrieval and knowledge · approach: Knowledge editing in LLMs · 12 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes LEARN, a twin-tower recommender that uses a frozen pretrained LLM (Baichuan2-7B) as an item content encoder plus a trainable 12-layer causal transformer that aligns content embeddings to the collaborative domain via contrastive recommendation loss, evaluated on an industrial short-video e-commerce dataset, online A/B tests, and six Amazon Review datasets.

Problem

Existing LLM-based recommenders adapt user-item interactions into text conversations ("Rec-to-LLM"), which is computationally unaffordable for long industrial user histories, causes catastrophic forgetting when finetuned, and mismatches training objectives, yielding poor practical benefit; ID-based

Approach

Reverse the adaptation direction ("LLM-to-Rec"): freeze the LLM and use it only as an item text encoder to get content embeddings, then project these into the collaborative recommendation space with a trainable transformer (PAL) supervised by a dense all-action contrastive recommendation loss over user history and target items, with recency-weighted sampling.

Key results

• On Amazon Review, LEARN improves Recall@10 over RecFormer by 10.08%, 17.87%, 5.39%, 10.41%, 29.45%, 10.50% on Scientific, Instruments, Arts, Office, Games, Pet.
• On the industry dataset, LLM-emb LEARN reaches H@100 0.0751 vs 0.0504 for ID embeddings (49.01% gain) and 0.0576 for BERT-emb.
• Online A/B test on a short-video e-commerce ranking model gives +0.84pp UAUC and +0.76pp WUAUC over baseline.

Measured effects reported in the paper

Surprising findings

• Replacing the 12-layer transformer PAL backbone with the pretrained LLM (Baichuan2-7B via LoRA, up to 572M trainable params) underperforms the small from-scratch transformer (H@100 0.0513 vs 0.0751).
• Using raw LLM content embeddings without alignment (w/o Align) performs very poorly, showing LLM embeddings are unsuitable for direct recommendation use.

Limitations

• Industry dataset is proprietary and results cannot be independently reproduced.
• Training uses truncated user sequences (history 80, target 40) due to memory limits.
• Amazon Review experiments use no sampling strategy and a single-target leave-one-out setting, so ItemTower variants coincide there.

Experimental setup

Datasets: Amazon Review (Scientific, Instruments, Arts, Office, Games, Pet, plus 7 pretraining categories) in-house short-video e-commerce interaction dataset

Benchmarks: Amazon Review six categories (NDCG@10, Recall@10, MRR) industry dataset (H@50, R@50, H@100, R@100) online A/B test (UAUC, WUAUC, CVR, revenue)

Metrics: NDCG@10 Recall@10 MRR Hit rate Recall UAUC WUAUC revenue

Baselines: GRU4Rec SASRec BERT4Rec RecGURU FDSA S3-Rec ZESRec UniSRec RecFormer

Builds on: RecFormer BERT dense all action loss LoRA

Closest papers by meaning

• Bridging the User-side Knowledge Gap in Knowledge-aware Recommendations with Large Language Models (AAAI 2025 · similarity 0.89)
• CoRA: Collaborative Information Perception by Large Language Model’s Weights for Recommendation (AAAI 2025 · similarity 0.89)
• Semantic Convergence: Harmonizing Recommender Systems via Two-Stage Alignment and Behavioral Semantic Tokenization (AAAI 2025 · similarity 0.89)
• Improving LLMs for Recommendation with Out-Of-Vocabulary Tokens (ICML 2025 · similarity 0.88)
• Can LLMs Outshine Conventional Recommenders? A Comparative Evaluation (NeurIPS 2025 · similarity 0.88)
• LLMRG: Improving Recommendations through Large Language Model Reasoning Graphs (AAAI 2024 · similarity 0.87)