Research map / Large language models / Instruction and fine-tuning
T-SHIRT: Token-Selective Hierarchical Data Selection for Instruction Tuning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes T-Shirt, a data selection framework for instruction tuning that scores responses with token-selective IFD (S-IFD) computed by GPT-2 and then picks samples with high neighborhood-average and low-variance quality under random embedding perturbations.
Problem
Sample-level LLM-based quality scores such as IFD treat all response tokens equally and are sensitive to small semantic-preserving input changes, so they can select samples for superficial rather than true quality reasons. The goal is cheap, robust selection of small instruction-tuning subsets.
Approach
Compute IFD only over the top-k% response tokens ranked by absolute instruction-conditioned log-likelihood shift (S-IFD), then rank samples hierarchically: keep the top gamma*b by mean S-IFD over uniform-noise embedding perturbations, and from those keep the b with lowest variance.
Key results
- On Alpaca-GPT-4 with 5% of data, T-Shirt improves the all-benchmark average by 5.10 to 5.48 points over full-dataset tuning, and reaches 57.60 average on Qwen-2.5-7B versus 56.81 for IFD.
- On Magpie with 10k samples for Qwen-2.5-7B, T-Shirt (k=50) achieves 58.45 overall average versus 58.05 for the strongest baseline IFD.
- Selecting 52k samples with GPT-2 S-IFD takes about 40 minutes on a single GPU, 2.7 to 3.7x faster than Deita and DS2.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| select 5% of Alpaca-GPT-4 using T-Shirt (k=75) instead of IFD | Llama-3.1-8B · Alpaca-GPT-4, eight-benchmark average | average score: 46.94 -> 47.71 | Helped |
| select 5% of Alpaca-GPT-4 using T-Shirt (k=50) instead of IFD | Qwen-2.5-7B · Alpaca-GPT-4, eight-benchmark average | average score: 56.81 -> 57.60 | Helped |
| select 5% of Alpaca-GPT-4 with T-Shirt rather than use the full dataset | Llama-3.1-8B and Qwen-2.5-7B · Alpaca-GPT-4, eight-benchmark average | average score: 5.10 to 5.48 points | Helped |
| select 10k Magpie samples using T-Shirt rather than IFD | Qwen-2.5-7B · Magpie, eight-benchmark average | average score: 58.05 -> 58.45 | Helped |
| select 10k Magpie samples using T-Shirt rather than IFD | Qwen-2.5-14B · Magpie, eight-benchmark average | average score: 63.12 -> 63.79 | Helped |
| add S-IFD to original IFD scoring (ablation) | T-Shirt data selection · Alpaca-GPT-4, OpenLLM average | average score: 60.01 -> 60.66; 70.48 -> 70.78 | Helped |
| add hierarchical selection to original IFD scoring (ablation) | T-Shirt data selection · Alpaca-GPT-4, OpenLLM average | average score: 60.01 -> 60.84; 70.48 -> 70.78 | Helped |
| combine S-IFD with hierarchical selection (ablation) | T-Shirt data selection · Alpaca-GPT-4, OpenLLM average | average score: 60.84 -> 60.91; 70.78 -> 70.91 | Helped |
| select low-variance rather than high-variance neighborhoods (ablation) | T-Shirt hierarchical selection · Alpaca-GPT-4 | OpenLLM average: 2.29 to 4.86 points | Helped |
| use binary token weights (1:0) rather than 1.5:1 or 2:1 weights (ablation) | T-Shirt S-IFD · Alpaca-GPT-4, OpenLLM average | average score: 70.54 -> 70.91; 70.56 -> 70.91 | Helped |
| vary token selection ratio (ablation) | T-Shirt S-IFD · Alpaca-GPT-4 | OpenLLM average | Mixed |
| reduce the number of perturbations M (ablation) | T-Shirt hierarchical selection · Alpaca-GPT-4 | OpenLLM average | No clear effect |
Surprising findings
- Full-dataset tuning on Alpaca-GPT-4 is outperformed by 5% selected subsets on the average metric, and the authors note Deita and DS2 underperform on GSM8k regardless of base model.
- Binary removal of uninformative tokens outperforms soft upweighting of informative tokens (1.5:1 and 2:1 ratios).
Limitations
- Experiments limited to 7B-14B models due to compute constraints.
- Judge models differ from official Arena-Hard and AlpacaEval setups due to API budget.
- Lacks a theoretical foundation for the hierarchical selection method.
Experimental setup
Datasets: Alpaca-GPT-4 Magpie-Pro-300K-Filtered ARC-Challenge HellaSwag MMLU TruthfulQA BBH GSM8k Arena-Hard AlpacaEval 2.0
Benchmarks: ARC-Challenge HellaSwag MMLU TruthfulQA BBH GSM8k Arena-Hard AlpacaEval 2.0
Metrics: normalized accuracy accuracy exact match style-controlled win rate length-controlled win rate average score
Baselines: Full Random Longest Deita DS2 IFD
Builds on: IFD Superfiltering LIMA NEFTune Deita
Closest papers by meaning
- NICE Data Selection for Instruction Tuning in LLMs with Non-differentiable Evaluation Metric (ICML 2025 · similarity 0.90)
- Long Is More for Alignment: A Simple but Tough-to-Beat Baseline for Instruction Fine-Tuning (ICML 2024 · similarity 0.89)
- LESS: Selecting Influential Data for Targeted Instruction Tuning (ICML 2024 · similarity 0.89)
- What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuning (ICLR 2024 · similarity 0.88)
- MAmmoTH2: Scaling Instructions from the Web (NeurIPS 2024 · similarity 0.88)
- AlpaGasus: Training a Better Alpaca with Fewer Data (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.