Research map / Large language models / Instruction and fine-tuning
DELIFT: Data Efficient Language model Instruction Fine-Tuning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DELIFT, a data selection framework that scores pairwise sample utility via an in-context-learning-based information gain and applies submodular optimization (FL, FLMI, FLCG) to select training subsets for instruction tuning, task-specific tuning, and continual fine-tuning.
Problem
Fine-tuning LLMs is expensive due to redundant data; existing selection methods either use costly gradient/influence estimates or static embeddings that do not track the model's evolving state, and usually target only one fine-tuning stage.
Approach
A pairwise utility metric, the change in the model's ground-truth likelihood when another sample is given as an in-context example (interpreted as conditional PMI), used as a kernel for stage-specific submodular selection (Facility Location, FLMI, FLCG) across instruction tuning, task-specific tuning, and continual tuning.
Key results
- On Mix-Instruct with Qwen2-72B, DELIFT with QLoRA reaches ROUGE 52.79 vs 65.51 for Full Data using 30% of data, beating LESS (45.16) and SelectIT (45.14).
- On HotpotQA to MMLU, DELIFT scores 81.70 for Qwen2 vs 78.36 for Full Data, exceeding full-data training.
- On IBM to Government with Qwen2 ICL, DELIFT reaches ROUGE 69.49 vs 66.08 for Full Data at 30% of data.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use DELIFT's utility-based kernel instead of static sentence-embedding similarities (ablation) | DELIFT · Mix-Instruct, Qwen2-72B-Instruct | LAJ: 3.28 -> 3.35 (ICL); 3.28 -> 3.37 (QLoRA) | Helped |
| use FLMI instead of FL for task alignment (ablation) | DELIFT · Mix-Instruct → GSM-8k, Mistral-7B-v0.1 | LAJ: 2.54 -> 2.56 (ICL); 2.63 -> 2.71 (QLoRA) | Helped |
| select 30% of Mix-Instruct with DELIFT rather than use the full dataset | Qwen2-72B-Instruct · Mix-Instruct | LAJ: 3.45 -> 3.35 (ICL); 3.51 -> 3.37 (QLoRA) | Hurt |
| select HotpotQA examples with DELIFT rather than use the full dataset | Qwen2 (QLoRA) · HotpotQA → MMLU (5-shot) | classification accuracy: 78.36 -> 81.70 | Helped |
| select HotpotQA examples with DELIFT rather than use the full dataset | Phi-3 (QLoRA) · HotpotQA → MMLU (5-shot) | classification accuracy: 64.50 -> 68.70 | Helped |
| select a 30% subset with DELIFT rather than use the full dataset | Qwen2-72B-Instruct · IBM → Government | LAJ: 3.65 -> 3.60 (ICL); 3.74 -> 3.65 (QLoRA) | Mixed |
| select a 30% subset with DELIFT rather than use the full dataset | Qwen2-72B-Instruct · SQuAD → HotpotQA | LAJ: 3.35 -> 3.33 (ICL); 3.48 -> 3.34 (QLoRA) | Hurt |
| increase subset size (ablation) | DELIFT · Use Cases 1–3 | LAJ | Helped |
| increase subset size beyond 50% (ablation) | DELIFT · Use Cases 1–3 | LAJ | No clear effect |
Surprising findings
- DELIFT matches or exceeds Full Data training in several settings (e.g., HotpotQA to MMLU), suggesting redundancy or harmful samples in the full data.
- The paper's own appendix states that evaluation is on Qwen2-72B-Instruct with Prometheus scores and that Euclidean distance is preferred over KL, departing from the PMI derivation.
Limitations
- O(n^2) cost of computing pairwise utility matrix limits scalability to very large datasets
- Sensitivity to quality and diversity of the initial data and risk of bias amplification
- Performance may vary across domains and multimodal settings
Experimental setup
Datasets: Mix-Instruct P3 HotpotQA SQuAD IBM query rewriting (proprietary)
Benchmarks: Mix-Instruct P3 HotpotQA MMLU MT-Bench GSM-8k SQuAD IBM and Government query rewriting
Metrics: ROUGE BGE similarity LAJ (LLM-as-judge score 1-5) classification accuracy
Baselines: Random SelectIT LESS DEFT-UCS DELIFT (SE) Full Data
Builds on: LESS SelectIT submodular data selection (facility location)
Closest papers by meaning
- SmallToLarge (S2L): Scalable Data Selection for Fine-tuning Large Language Models by Summarizing Training Trajectories of Small Models (NeurIPS 2024 · similarity 0.91)
- Two-stage LLM Fine-tuning with Less Specialization and More Generalization (ICLR 2024 · similarity 0.90)
- Enhancing Large Language Model Performance with Gradient-Based Parameter Selection (AAAI 2025 · similarity 0.90)
- LensLLM: Unveiling Fine-Tuning Dynamics for LLM Selection (ICML 2025 · similarity 0.90)
- When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method (ICLR 2024 · similarity 0.90)
- Data Mixing Optimization for Supervised Fine-Tuning of Large Language Models (ICML 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.