Research map / Large language models / Instruction and fine-tuning
Data Efficient Adaptation in Large Language Models via Continuous Low-Rank Fine-Tuning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DEAL, a LoRA-based continual fine-tuning framework that uses a wavelet/heat-kernel knowledge retention module and a regularized MLP-based update module to learn sequential tasks with small data.
Problem
Continual fine-tuning of LLMs with LoRA suffers from catastrophic forgetting and poor data efficiency when adapting sequentially to new tasks with small datasets, with existing model-editing and adapter-stacking methods being costly or opaque.
Approach
Applies a heat-kernel wavelet filter over truncated-SVD-like singular structure of LoRA matrices A and B to retain core historical features, and an MLP with asymmetric regularization orders (a ≥ b) to control updates to LoRA matrices, with the updated matrices simply substituted back into LoRA so inference cost is unchanged.
Key results
- On the 4-task benchmark with T5-Large, DEAL reaches 78.5% AA and 82.5% R-1, versus 44.6% for SeqLoRA and 71.2% for O-LoRA.
- On the 15-task benchmark with LLaMA-3.1-8B, DEAL reaches 74.6% AA, above SeqLoRA (45.2%) and O-LoRA (73.2%) but below PerTaskFT (77.1%).
- Training throughput drops from 31.62 to 17.88 samples/sec versus LoRA on DBpedia with T5-large, while inference latency is nearly unchanged (71.89 vs 73.32 ms/sample).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DEAL instead of O-LoRA | T5-Large continual learning · 4-Task (Standard) | AA: 71.2 -> 78.5 | Helped |
| Use DEAL instead of O-LoRA | T5-Large continual learning · 4-Task (Standard) | R-1: 73.3 -> 82.5 | Helped |
| Jointly update Adapter A and Adapter B rather than updating one branch only (ablation) | DEAL · continual learning, post-task | AA: 2.8–-5.4 percentage points | Helped |
| Use Adapter A rather than Adapter B alone (ablation) | DEAL · continual learning | AA | Helped |
| Increase LoRA rank from 4 to 8 (ablation) | DEAL · 3-task benchmark | accuracy: 71.5% to 84.3% | Helped |
| Increase LoRA rank from 8 to 16 or 32 (ablation) | DEAL · 3-task benchmark | accuracy: 84.3% ... 84.5% at rank 16, 84.6% at rank 32 | No clear effect |
| Set regularization weights to (a=10, b=2) rather than (1,1) (ablation) | DEAL · 4-task benchmark | AA: 74.8 -> 85.5 | Helped |
| Set regularization weights to (a=10, b=2) rather than (10,5) (ablation) | DEAL · 4-task benchmark | AA: 84.1 -> 85.5 | Helped |
| Set regularization weights to (a=10, b=2) rather than (20,2) (ablation) | DEAL · 4-task benchmark | AA: 82.7 -> 85.5 | Helped |
| Vary the task order across three permutations (ablation) | DEAL · 4-task benchmark | AA: 73.1% to 75.6% | No clear effect |
| Replace the heat kernel with f(x)=x e^{-x} (ablation) | DEAL · 4-task benchmark | AA: 78.4 vs 78.5 | No clear effect |
| Replace the heat kernel with quadratic splines (ablation) | DEAL · 4-task benchmark | AA: 78.3 vs 78.5 | No clear effect |
Surprising findings
- The paper reports DEAL below the PerTaskFT oracle on the 15-task (74.6 vs 77.1 AA) and 3-task T5 (87.7 vs 90.3 AA) settings, while the text describes performance as comparable.
- Updating only Adapter A consistently outperforms updating only Adapter B, which the authors attribute to A capturing generalizable directions.
Limitations
- Assumes a fixed task order and static model capacity, as stated by the authors.
- Training throughput is about 43% lower than standard LoRA with slightly higher training memory.
Experimental setup
Datasets: AG News DBpedia Yahoo Answers Amazon Reviews MNLI QQP RTE SST-2 WiC CB COPA MultiRC
Benchmarks: AG News/DBpedia/Yahoo/Amazon continual benchmark (3-task TC) 4-task standard CL benchmark (O-LoRA setting) 15-task large-scale benchmark
Metrics: Average Accuracy (AA) ROUGE-1
Baselines: SeqLoRA O-LoRA PerTaskFT SeqSVD Replay EWC LwF L2P ProgPrompt LB-CL
Builds on: LoRA O-LoRA
Closest papers by meaning
- Beyond LoRA vs. Full Fine-Tuning: Gradient-Guided Optimizer Routing for LLM Adaptation (NeurIPS 2026 · similarity 0.93)
- Gated Integration of Low-Rank Adaptation for Continual Learning of Large Language Models (NeurIPS 2025 · similarity 0.92)
- Bayesian Low-rank Adaptation for Large Language Models (ICLR 2024 · similarity 0.91)
- Loquetier: A Virtualized Multi-LoRA Framework for Unified LLM Fine-tuning and Serving (NeurIPS 2025 · similarity 0.91)
- GoRA: Gradient-driven Adaptive Low Rank Adaptation (NeurIPS 2025 · similarity 0.91)
- When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method (ICLR 2024 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.