Research map / Large language models / Reasoning and chain of thought
GREATS: Online Selection of High-Quality Data for LLM Training in Every Iteration
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GREATS, an online batch selection method that greedily selects training batch subsets using a Taylor-expansion approximation of validation-loss reduction, made practical by a "ghost inner-product" technique for computing per-sample gradient inner-products in one backpropagation.
Problem
Online batch selection for LLM training is either impractical (reference-model methods) or relies on heuristics (high-loss or gradient-norm methods) that may not capture true data informativeness; efficient, principled selection at training speed is needed.
Approach
Approximate the one-step validation-loss reduction of a candidate subset with first-order gradient inner-products (TracIN-style) corrected by a pairwise Hessian/identity interaction term, then run a greedy selection over each batch; compute all inner-products in a single backward pass using ghost inner-product tricks borrowed from differential privacy.
Key results
- On MMLU (9 subjects), GREATS reaches 54.2% average accuracy vs 50.4% for regular training and 50.8% for MaxLoss, and 55.0% vs 54.3% regular on TyDiQA.
- GREATS with ghost inner-product runs at 71.3 points/s vs 76.2 for regular training, while direct per-sample implementation runs at 4.2 points/s.
- GREATS gives faster test perplexity reduction than baselines across fine-tuning settings and provides marginal gains in GPT-small pretraining on OpenWebText.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GREATS online batch selection instead of regular training | Llama-2-7B · MMLU (9 subjects, average) | accuracy: 50.4% -> 54.2% | Helped |
| Use GREATS online batch selection instead of GradNorm | Llama-2-7B · MMLU (9 subjects, average) | accuracy: 50.4% -> 54.2% | Helped |
| Use GREATS online batch selection instead of MaxLoss | Llama-2-7B · MMLU (9 subjects, average) | accuracy: 50.8% -> 54.2% | Helped |
| Use GREATS online batch selection instead of regular training | Mistral-7B · TYDIQA test set | accuracy: 54.3% -> 55.0% | Helped |
| Use GREATS online batch selection instead of GradNorm | Mistral-7B · TYDIQA test set | accuracy: 53.4% -> 55.0% | Helped |
| Use GREATS online batch selection instead of MaxLoss | Mistral-7B · TYDIQA test set | accuracy: 54.7% -> 55.0% | Helped |
| Use ghost inner-product computation instead of direct per-sample gradients (ablation) | GREATS on GPT-SMALL · OPENWEBTEXT | throughput (training data points per second): 4.2 -> 71.3 | Helped |
| Use GREATS with ghost inner-product instead of regular training | GPT-SMALL · OPENWEBTEXT | throughput (training data points per second): 76.2 -> 71.3 | Hurt |
| Use GREATS with only 2 validation examples (ablation) | GREATS · MMLU test set | perplexity | Helped |
| Use GREATS for pretraining with validation data from the training distribution (ablation) | GPT-SMALL · OPENWEBTEXT test set | perplexity | Helped |
Surprising findings
- Validation-free heuristics GradNorm and MaxLoss can underperform regular uniform training, plausibly by selecting corrupted or unlearnable data.
- GREATS remains beneficial with as few as 2 validation examples.
Limitations
- Requires a validation set available before training.
- Ghost inner-product technique is tailored to SGD and not directly extendable to Adam; SGD used as a proxy.
- Validation loss/perplexity is the objective and gains on downstream tasks are often minor.
Experimental setup
Datasets: LESS Alpaca OpenWebText MMLU TyDiQA SAMSum
Benchmarks: MMLU TyDiQA SAMSum OpenWebText
Metrics: perplexity accuracy F1 throughput
Baselines: Regular training MaxLoss GradNorm RHOLoss SBERT distance selection
Builds on: TracIN ghost clipping LESS RHO-Loss
Closest papers by meaning
- BLADE: Scalable Bi-level Adaptive Data Selection for LLM Training (NeurIPS 2026 · similarity 0.91)
- Rethinking Data Curation in LLM Training: Online Reweighting Offers Better Generalization than Offline Methods (ICLR 2026 · similarity 0.89)
- Data Selection via Optimal Control for Language Models (ICLR 2025 · similarity 0.88)
- LLM Data Selection and Utilization via Dynamic Bi-level Optimization (ICML 2025 · similarity 0.88)
- Efficient Data Selection at Scale via Influence Distillation (NeurIPS 2025 · similarity 0.88)
- DsDm: Model-Aware Dataset Selection with Datamodels (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.