Research map / Large language models / Instruction and fine-tuning
SHED: Shapley-Based Automated Dataset Refinement for Instruction Fine-Tuning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SHED, a Shapley-value-based dataset refinement framework that clusters data, computes approximate Shapley values only for cluster proxies, and samples compact high-quality instruction-tuning subsets that transfer across LLMs.
Problem
Selecting small, high-quality instruction fine-tuning datasets from large pools while accounting for combinatorial interactions between samples; exact Shapley values are exponentially expensive.
Approach
Cluster data with model-agnostic sentence embeddings, compute Shapley values only for one proxy per cluster using a grouped-removal approximation, and use cluster Shapley scores to drive quality-ordered (QOCS) or quality-weighted (QWCS) sampling.
Key results
- On MMLU, SHED-QOCS best-selected subset (10k) reaches 44.80 vs 45.56 for full data in the best-selected table, while the text claims 2.76% higher accuracy than full dataset; the table reports full at 45.56 so the claim is inconsistent with the table.
- On ARC-challenge with MMLU source, SHED-QWCS (9k) reaches 49.21 vs 45.99 for full 99.8k dataset.
- On ARC-challenge with WizardLM source, SHED-QOCS (1k) reaches 51.36 vs 47.95 for full 70k dataset.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SHED-QOCS-selected WizardLM data instead of the full WizardLM dataset | LLaMA-7B · MMLU test | accuracy: 33.16 -> 35.92 | Helped |
| Use SHED-QOCS-selected MMLU data instead of the full MMLU dataset | LLaMA-7B · ARC-challenge | accuracy: 45.99 -> 47.10 | Helped |
| Use SHED-QWCS-selected MMLU data instead of the full MMLU dataset | LLaMA-7B · ARC-challenge | accuracy: 45.99 -> 49.21 | Helped |
| Use SHED-QOCS-selected WizardLM data instead of the full WizardLM dataset | LLaMA-7B · ARC-challenge | accuracy: 47.95 -> 51.36 | Helped |
| Use SHED-QOCS-selected MMLU data instead of the full MMLU dataset | LLaMA-7B · MMLU test | accuracy: 45.56 -> 44.80 | Hurt |
| Use SHED-QWCS-selected WizardLM data instead of the full WizardLM dataset | LLaMA-7B · MMLU test | accuracy: 33.16 -> 35.76 | Helped |
| Use SHED-QWCS-selected WizardLM data instead of the full WizardLM dataset | LLaMA-7B · ARC-challenge | accuracy: 47.95 -> 50.26 | Helped |
| Use SHED-QWCS instead of SHED-QOCS | LLaMA-7B fine-tuned on WizardLM-selected data · MT-Bench | MT-Bench score: 4.89 -> 5.24 | Helped |
| Use SHED-selected 1k WizardLM samples instead of IFD-selected 7k samples | LLaMA-7B · MMLU | accuracy: 33.08 -> 33.62 | Helped |
| Use SHED-selected 1k WizardLM samples instead of IFD-selected 7k samples | LLaMA-7B · ARC-challenge | accuracy: 52.90 -> 51.36 | Hurt |
| Vary the number of K-means clusters (ablation) | SHED-QOCS and SHED-QWCS · MMLU test | accuracy | Mixed |
| Increase Shapley-calculation iterations beyond 10 (ablation) | SHED-QOCS and SHED-QWCS · MMLU test | accuracy | No clear effect |
Surprising findings
- Naively combining individually high-quality samples (DSIR high-quality, 40.04) gives little gain over random (39.13), motivating combination-aware selection.
- Subsets selected with LLaMA-7B transfer to LLaMA-13B, Vicuna-7B, and GPT-2 with competitive performance.
Limitations
- Relies on representative sentence embeddings; may be inadequate where embeddings are poor.
- Clustering and proxy data may overlook rare but important samples.
- Current objective focuses on accuracy and does not address fairness or bias (fairness only discussed as a possible extension).
Experimental setup
Datasets: MMLU WizardLM-evol-instruct-70k
Benchmarks: MMLU ARC-challenge MT-Bench
Metrics: accuracy MT-Bench score
Baselines: Random sampling Dataset Quantization DSIR Full dataset fine-tuning LIMA IFD
Builds on: Shapley value DSIR LoRA TS-DShapley K-means
Closest papers by meaning
- MAmmoTH2: Scaling Instructions from the Web (NeurIPS 2024 · similarity 0.90)
- PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimization (ICLR 2024 · similarity 0.89)
- LESS: Selecting Influential Data for Targeted Instruction Tuning (ICML 2024 · similarity 0.89)
- What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuning (ICLR 2024 · similarity 0.88)
- Long Is More for Alignment: A Simple but Tough-to-Beat Baseline for Instruction Fine-Tuning (ICML 2024 · similarity 0.88)
- Automating Dataset Updates Towards Reliable and Timely Evaluation of Large Language Models (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.