Research map / Large language models / Language, safety and interpretability
Multimodal Quantitative Language for Generative Recommendation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Translates item text and image content into a unified quantitative language via RQ-VAE quantizers, then pre-trains and fine-tunes a T5-style encoder-decoder with multimodal quantitative language generation tasks for generative sequential recommendation.
Problem
Generative recommendation built on pre-trained language models suffers from a mismatch between general linguistic knowledge and recommendation needs, and it rarely exploits complementary knowledge across item modalities (text and image).
Approach
Quantize text (LLaMA) and image (CLIP ViT-L/14) item embeddings with separate RQ-VAE translators into a shared, concise token vocabulary, then design next-item, asymmetric-item, and cross-modal alignment generation tasks with prompt tokens, and transfer knowledge from source-domain pre-training; collisions are resolved by distance-based token reallocation.
Key results
- On Instruments, MQL4GRec reaches NDCG@10 0.1060 versus 0.0950 for TIGER (+11.58%), with improvements of 14.82% and 7.95% on Arts and Games (as stated in the abstract, where the NDCG@10 gains are 12.56% and 7.66% in the table).
- Pre-training with NIG1 raised Instruments NDCG@10 from 0.0987 to 0.1043 and Arts NDCG@10 from 0.0844 to 0.0959 before re-ranking.
- Zero-shot pre-trained models show weak but non-trivial HR@10 on Instruments (0.00345) and Arts (0.00311), and near-zero on Games.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MQL4GRec instead of the strongest baseline | Generative recommendation · Instruments | NDCG@10: 0.0950 -> 0.1060 | Helped |
| Use MQL4GRec instead of the strongest baseline | Generative recommendation · Arts | NDCG@10: 0.0844 -> 0.0950 | Helped |
| Use MQL4GRec instead of the strongest baseline | Generative recommendation · Games | NDCG@10: 0.0499 -> 0.0548 | Helped |
| Add re-ranking of recommendations from the text and image next-item tasks (ablation) | MQL4GRec · Instruments | NDCG@10: 0.1051 -> 0.1060 | Helped |
| Add re-ranking of recommendations from the text and image next-item tasks (ablation) | MQL4GRec · Arts | NDCG@10: 0.0944 -> 0.0950 | Helped |
| Add re-ranking of recommendations from the text and image next-item tasks (ablation) | MQL4GRec · Games | NDCG@10: 0.0521 -> 0.0548 | Helped |
| Add quantitative language generation tasks beyond text-only next-item generation (ablation) | Text quantitative language · Arts | NDCG@10: 0.0844 -> 0.0913 | Helped |
| Add asymmetric item generation and quantitative language alignment (ablation) | Text quantitative language · Games | NDCG@10: 0.0473 -> 0.0531 | Helped |
| Add quantitative language alignment to NIG + AIG (ablation) | Text quantitative language · Instruments | NDCG@10: 0.0987 -> 0.0993 | Helped |
| Add source-domain pre-training to QLG (ablation) | Text quantitative language · Instruments | NDCG@10: 0.0993 -> 0.1051 | Helped |
| Add source-domain pre-training to QLG (ablation) | Text quantitative language · Arts | NDCG@10: 0.0913 -> 0.0944 | Helped |
| Add source-domain pre-training to QLG (ablation) | Text quantitative language · Games | NDCG@10: 0.0531 -> 0.0521 | Hurt |
Surprising findings
- Pre-training on source domains degraded Games performance when image quantitative language was used, which the authors attribute to overfitting or domain conflict.
- Including a user ID token in the TIGER input degraded performance on Games, attributed to user ID collisions in a 2000-token space.
Limitations
- Inference is slower than traditional recommenders because of autoregressive beam search.
- Requires item text and image content; missing-content scenarios are not studied.
- Pre-training transfer is inconsistent across datasets (degrades on Games).
Experimental setup
Datasets: Amazon Product Reviews (Pet Supplies, Cell Phones and Accessories, Automotive, Tools and Home Improvement, Toys and Game Amazon Musical Instruments Amazon Arts Crafts and Sewing Amazon Video Games
Benchmarks: Amazon Musical Instruments Amazon Arts Crafts and Sewing Amazon Video Games
Metrics: HR@K NDCG@K (K=1,5,10)
Baselines: GRU4Rec BERT4Rec SASRec FDSA S3-Rec VQ-Rec MISSRec P5-CID VIP5 TIGER
Builds on: TIGER RQ-VAE T5 P5 LC-Rec
Closest papers by meaning
- Harnessing Multimodal Large Language Models for Multimodal Sequential Recommendation (AAAI 2025 · similarity 0.89)
- Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations (ICML 2024 · similarity 0.88)
- Towards Unified Multi-Modal Personalization: Large Vision-Language Models for Generative Recommendation and Beyond (ICLR 2024 · similarity 0.87)
- Language Representations Can be What Recommenders Need: Findings and Potentials (ICLR 2025 · similarity 0.87)
- Can LLMs Outshine Conventional Recommenders? A Comparative Evaluation (NeurIPS 2025 · similarity 0.86)
- Curriculum Conditioned Diffusion for Multimodal Recommendation (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.