Research map / Large language models / Instruction and fine-tuning

Data Mixing Optimization for Supervised Fine-Tuning of Large Language Models

ICML 2025 · Large language models › Instruction and fine-tuning · approach: Efficient LLM fine-tuning systems · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Frames SFT data mixing as minimizing validation loss via a parametric loss that models effective transferred data and a fine-tuning power law, then solves for domain weights with SLSQP.

Problem

Choosing domain weights for the data mixture in supervised fine-tuning (SFT) of LLMs is underexplored; pre-training data-mixing methods transfer poorly to SFT and can give unbalanced domain weights when data scales.

Approach

Parametrize per-domain validation loss as a power law in effective data (own tokens plus transferred tokens from other domains, modeled as k*|D_other|^alpha), fit five parameters per domain from five perturbation runs with small data, then minimize the summed loss over the simplex with SLSQP; proofs of convexity and a "no domains left behind" property are given.

Key results

• On 3-domain mixing across four models and three budgets, the method's overall PPL averages only 0.66% above the best grid-search mixture.
• On Tulu3 re-weighting with Llama3.1-8B, the method achieves the lowest average validation PPL (2.150 vs 2.162 original) and the best downstream average for Llama3.1-8B and Llama3.1-70B (50.4 and 63.8).
• In a medical case study, the optimized mixture (67.7% Alpaca-GPT4, 32.3% PubMedQA) gives PPL 5.42 vs 5.48 for all-medical data.

Measured effects reported in the paper

Surprising findings

• Optimal domain weights depend on data budget (scale-dependent), contrary to the scale-invariance assumed by many prior methods.
• For a purely medical target, the method recommends a majority of general instruction-following data, beating training on medical data alone.

Limitations

• Downstream gains on Orca do not track validation loss, and IFEval drops for Llama models under the Tulu3 weights.
• Repeated upsampling of limited data for under-represented domains (e.g., precise IF) may cause overfitting.
• Domains are defined by data provenance, and the method is agnostic to within-domain composition.

Experimental setup

Datasets: Infinity-Instruct OpenMathInstruct-2 OpenCoder (opc-sft-stage1) Tulu3 SFT mixture OpenOrca Alpaca-GPT4 PubMedQA

Benchmarks: AGIEval IFEval MMLU GSM8K HumanEval ToxiGen TruthfulQA HellaSwag

Metrics: token-level perplexity per-domain perplexity accuracy

Baselines: Grid search mixtures Original Equal_T Equal_I SMART

Builds on: DoReMi Data Mixing Laws RegMix AutoScale

Closest papers by meaning

• FisherSFT: Data-Efficient Supervised Fine-Tuning of Language Models Using Information Gain (ICML 2025 · similarity 0.93)
• SmallToLarge (S2L): Scalable Data Selection for Fine-tuning Large Language Models by Summarizing Training Trajectories of Small Models (NeurIPS 2024 · similarity 0.91)
• Data Efficient Adaptation in Large Language Models via Continuous Low-Rank Fine-Tuning (NeurIPS 2025 · similarity 0.91)
• Get more for less: Principled Data Selection for Warming Up Fine-Tuning in LLMs (ICLR 2024 · similarity 0.91)
• Token Cleaning: Fine-Grained Data Selection for LLM Supervised Fine-Tuning (ICML 2025 · similarity 0.90)
• Data Mixing Laws: Optimizing Data Mixtures by Predicting Language Modeling Performance (ICLR 2025 · similarity 0.90)