Research map / Continual learning / Continual fine-tuning of LLMs
Meta-UCF: Unified Task-Conditioned LoRA Generation for Continual Learning in Large Language Models
ShiLin Xiao, Tianxiang Xu, Canran Xiao, Weihao Luo, Liwei Hou, Chuangxin Zhao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add orthogonality penalty Lorth (ablation) | Meta-UCF · Std-CL 5 | average accuracy: 83.9 -> 85.6 | Improved |
| add orthogonality penalty Lorth (ablation) | Meta-UCF · Long-CL 15 | average accuracy: 68.5 -> 70.7 | Improved |
| add meta-contrastive loss Lctr (ablation) | Meta-UCF · Std-CL 5 | average accuracy: 84.1 -> 85.6 | Improved |
| add meta-contrastive loss Lctr (ablation) | Meta-UCF · Long-CL 15 | forgetting ratio: 12.7 -> 11.5 | Improved |
| add bias calibration (ablation) | Meta-UCF · Long-CL 15 | average accuracy: 69.4 -> 70.7 | Improved |
| use layer-normalised mean pooling instead of last CLS (ablation) | Meta-UCF · Std-CL 5 | average accuracy: 82.1 -> 85.6 | Improved |
| add task-conditioned generator instead of static LoRA (ablation) | Meta-UCF · Long-CL 15 | average accuracy: 64.9 -> 70.7 | Improved |
| use historical replay support instead of current-task-only support (ablation) | Meta-UCF · Std-CL 5 | average accuracy: 83.4 -> 85.6 | Improved |
| increase replay buffer from 256 to 512 (ablation) | Meta-UCF · Long-CL 15 | forgetting ratio: 11.5 -> 10.9 | Improved |
| reduce LoRA rank from 8 to 4 (ablation) | Meta-UCF · Long-CL 15 | average accuracy: 70.7 -> 69.0 | Hurt |
| increase LoRA rank from 8 to 16 (ablation) | Meta-UCF · Long-CL 15 | average accuracy | No clear effect |
| restrict LoRA to upper half of layers (ablation) | Meta-UCF · Std-CL 5 | average accuracy: 85.6 -> 84.9 | Mixed |
Closest papers by meaning
- Gated Integration of Low-Rank Adaptation for Continual Learning of Large Language Models (NeurIPS 2025 · similarity 0.91)
- Estimating and Orthogonalizing Unknown Pre-training Gradients for Continual Fine-tuning of Large Language Models (NeurIPS 2026 · similarity 0.90)
- Learn more, but bother less: parameter efficient continual learning (NeurIPS 2024 · similarity 0.90)
- Scalable Language Model with Generalized Continual Learning (ICLR 2024 · similarity 0.89)
- Data Efficient Adaptation in Large Language Models via Continuous Low-Rank Fine-Tuning (NeurIPS 2025 · similarity 0.89)
- Beyond LoRA vs. Full Fine-Tuning: Gradient-Guided Optimizer Routing for LLM Adaptation (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
