Research map / Continual learning / Continual fine-tuning of LLMs
Continual Low-Rank Adapters for LLM-based Generative Recommender Systems
Hyunsik Yoo, Ting-Wei Li, SeongKu Kang, Zhining Liu, Charlie Xu, Qilin Qi, Hanghang Tong
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add proximal regularizer toward previous LoRA state | single evolving LoRA · Instruments | NDCG@5: 0.0127 -> 0.0138 | Improved |
| add proximal regularizer toward previous LoRA state | single evolving LoRA · Books | Hit@5: 0.0448 -> 0.0448 | No clear effect |
| sum all frozen past adapters (cumulative LoRA) without parameter inheritance | single evolving LoRA · Amazon Instruments | NDCG@5: 0.0127 -> 0.0093 | Hurt |
| add parameter inheritance to cumulative LoRA | SumLoRA / cumulative · Instruments natural split | relative NDCG@5 gain vs single: -26.77% -> 1.57% | Improved |
| use user-disjoint split instead of natural chronological split | cumulative LoRA (all) · Instruments | relative NDCG@5 gain vs single: -26.77% -> -8.13% | Improved |
| learnable magnitudes for past adapters (SD-LoRA) vs fixed | cumulative LoRA latest+inherit · Instruments | NDCG@5: 0.0130 -> 0.0128 | No clear effect |
| add orthogonality regularizer against previous adapter | PESO · regularizer comparison | performance | Hurt |
| use L2 proximal instead of softmax-KL proximal (ablation) | PESO · regularizer comparison | performance | Hurt |
| use LoRA-output KL or per-rank KL instead of per-module softmax-KL (ablation) | PESO · regularizer comparison | performance | No clear effect |
| increase proximal weight lambda from 0 (ablation) | PESO · lambda sweep | average of four metrics | Mixed |
| use pretraining learning rate on incremental blocks (lr*=1.0) | single evolving LoRA · incremental stages | performance | Hurt |
| scale learning rate to 0.05-0.1x for incremental blocks | single evolving LoRA · incremental stages | performance | Improved |
Closest papers by meaning
- Customizing Language Models with Instance-wise LoRA for Sequential Recommendation (NeurIPS 2024 · similarity 0.88)
- Gated Integration of Low-Rank Adaptation for Continual Learning of Large Language Models (NeurIPS 2025 · similarity 0.86)
- On Efficiency-Effectiveness Trade-off of Diffusion-based Recommenders (NeurIPS 2025 · similarity 0.85)
- Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations (ICML 2024 · similarity 0.85)
- Influential Exemplar Replay for Incremental Learning in Recommender Systems (AAAI 2024 · similarity 0.85)
- CoRA: Collaborative Information Perception by Large Language Model’s Weights for Recommendation (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
