Research map / Continual learning / Continual fine-tuning of LLMs
DRIVE: Fine-tuning via Data Contribution- and Diversity-aware Weighting with Prior Regularization
qing liu, Xinrui Chen, Weiyao Zhu, Yi Du, Ou Wu
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DRIVE, a data-weighting fine-tuning method that uses contribution- and diversity-aware sample weights with prior regularization to reduce catastrophic forgetting when source pre-training data is inaccessible.
Problem
Single-stage fine-tuning causes catastrophic forgetting of pre-trained knowledge when the source pre-training data is unavailable; existing data weighting methods lack a principled way to trade off knowledge retention against new-task adaptation.
Approach
Weights fine-tuning samples by their contribution and diversity and adds a prior regularization term, giving a unified differentiable objective that can accommodate auxiliary signals; the abstract indicates this explicitly arbitrates retention versus adaptation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| remove task-adaptation component (lambda=1) (ablation) | DRIVE · MetaMathQA, Llama-3.1-8B | target accuracy: 77.03 -> 73.19 | Improved |
| remove anti-forgetting component (lambda=0) (ablation) | DRIVE · MetaMathQA, Llama-3.1-8B | general capability: 57.51 -> 52.24 | Improved |
| remove diversity regularization (eta=0) (ablation) | DRIVE · MetaMathQA, Llama-3.1-8B | overall accuracy: 67.27 -> 65.48 | Improved |
| remove uniform KL prior regularization (ablation) | DRIVE · MetaMathQA, Llama-3.1-8B | overall accuracy: 67.27 -> 62.29 | Improved |
| replace differentiable diversity regularizer with discrete clustering (DRIVE-Cluster) (ablation) | DRIVE · MetaMathQA, Llama-3.1-8B | overall accuracy | Improved |
| use entropy-informed prior instead of uniform prior (ablation) | DRIVE · MetaMathQA, Llama-3.1-8B | general/overall accuracy | No clear effect |
| decay lambda from 1 to 0 instead of best static lambda=0.8 (ablation) | DRIVE · ResNet-50 vision | average accuracy | Improved |
| increase eta beyond 1.0 (ablation) | DRIVE · hyperparameter sensitivity | generalization | Hurt |
| use gradient-norm contribution scores instead of loss (ablation) | DRIVE · Llama-3.2-3B | overall accuracy: 57.69 -> 57.99 at 2.12x cost | Mixed |
| use entropy-based contribution scores instead of loss (ablation) | DRIVE · Llama-3.2-3B | overall accuracy: 57.69 -> 55.14 | Hurt |
| shuffle contribution score ranking (ablation) | DRIVE · MetaMathQA, Llama-3.1-8B | overall accuracy: 67.29 -> 65.12 | Hurt |
| reverse contribution score ranking (ablation) | DRIVE · MetaMathQA, Llama-3.1-8B | overall accuracy: 67.29 -> 60.23 | Hurt |
Closest papers by meaning
- Upweighting Easy Samples in Fine-Tuning Mitigates Forgetting (ICML 2025 · similarity 0.90)
- LIFT+: Lightweight Fine-Tuning for Long-Tail Learning (TPAMI 2026 · similarity 0.87)
- RECAST: Reparameterized, Compact weight Adaptation for Sequential Tasks (ICLR 2025 · similarity 0.85)
- Pay Attention to Small Weights (NeurIPS 2025 · similarity 0.85)
- LEVI: Generalizable Fine-tuning via Layer-wise Ensemble of Different Views (ICML 2024 · similarity 0.85)
- Mitigating Forgetting in LLM Fine-Tuning via Low-Perplexity Token Learning (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
