Research map / Large language models / Instruction and fine-tuning

Online Data Selection for Instruction Tuning via Gaussian Processes

NeurIPS 2026 · Large language models › Instruction and fine-tuning · approach: Efficient LLM fine-tuning systems · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes GAIA, which models training-set quality with a Gaussian process over discretized strategies and updates strategy weights with a fixed-share Hedge rule, so online batch sampling is guided globally rather than by random candidate pools.

Problem

Online data selection for LLM instruction tuning is batch-constrained: selection only refines randomly drawn candidate pools, so its quality is capped by pool randomness.

Approach

Use a GP prior over quality scores, discretize it into paired function samples (strategies), sample candidate batches via softmax over a strategy's scores, and update strategy posteriors with a top-k likelihood under fixed-share Hedge, which gives a dynamic-regret bound.

Key results

• On MMLU-Sociology with Llama-2-7b, GAIA-GREATS reaches 66.50% ± 1.92 accuracy versus 64.34% ± 1.02 for GREATS (+3.57%).
• Minimum test perplexity on MMLU-Sociology is 3.75 ± 0.33 versus 4.37 ± 0.64 for GREATS, a reported 14.13% reduction.
• Across five MMLU subjects, GAIA average test perplexity is 5.9201 versus 6.6981 for GREATS and 7.8854 for regular training.

Surprising findings

• Skipping the batch refinement step (w/o BR) performs better than the full pipeline, which the authors read as evidence that global sampling itself finds high-quality data.
• On TyDiQA, GAIA-GREATS converges more slowly early on and only reaches a marginally better perplexity floor, with F1 on par with GREATS.

Limitations

• Performance is bounded by the fidelity of the base scoring function, acting as an amplifier rather than a corrective mechanism.
• Sensitive to hyperparameters such as strategy pool size m and sampling temperature tau, which need dataset-specific calibration.
• Evaluation limited to instruction tuning; pre-training not explored.

Experimental setup

Datasets: LESS Alpaca MMLU SamSum TyDiQA

Benchmarks: MMLU SamSum TyDiQA

Metrics: perplexity accuracy ROUGE F1

Baselines: GREATS GradNorm MaxLoss RHOLoss SBERT Regular full-data training

Builds on: GREATS fixed-share Hedge LESS

Closest papers by meaning

• NICE Data Selection for Instruction Tuning in LLMs with Non-differentiable Evaluation Metric (ICML 2025 · similarity 0.88)
• SelectIT: Selective Instruction Tuning for LLMs via Uncertainty-Aware Self-Reflection (NeurIPS 2024 · similarity 0.88)
• MAmmoTH2: Scaling Instructions from the Web (NeurIPS 2024 · similarity 0.87)
• LESS: Selecting Influential Data for Targeted Instruction Tuning (ICML 2024 · similarity 0.87)
• PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimization (ICLR 2024 · similarity 0.87)
• Adapt-$\infty$: Scalable Continual Multimodal Instruction Tuning via Dynamic Data Selection (ICLR 2025 · similarity 0.87)