AI Paper Atlas

Research map / Large language models / Instruction and fine-tuning

Utility-Diversity Aware Online Batch Selection for LLM Supervised Fine-tuning

Heming Zou, Yixiu Mao, Yun Qu, Qi Wang, Xiangyang Ji

ICML 2026 · Large language models › Instruction and fine-tuning · approach: Fine-tuning generalization and delta compression · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use UDS (nuclear norm plus diversity distance) instead of GREATS for online batch selectionQwen-2.5-7B · MMLUaverage accuracy (%): 58.19 -> 63.34Improved
Use UDS instead of full-dataset SFTQwen-2.5-7B · MMLUaverage accuracy (%): 55.32 -> 63.34Improved
Use UDS instead of full-dataset SFTQwen-2.5-7B · MMLU training throughputsamples per second: 2.27 -> 3.41Improved
Use UDS instead of full-dataset SFTQwen-2.5-7B · ScienceQAaverage accuracy (%): 94.56 -> 95.19Improved
Add nuclear norm score to random selection (ablation)Random selection baseline · MMLUaverage accuracy (%): 54.26 -> 58.35Improved
Add diversity distance to random selection (ablation)Random selection baseline · MMLUaverage accuracy (%): 54.26 -> 57.75Improved
Combine diversity distance with nuclear norm (ablation)UDS nuclear-norm-only · MMLUaverage accuracy (%): 58.35 -> 63.34Improved
Combine diversity distance with nuclear norm (ablation)UDS nuclear-norm-only · HumanEvalPass@1 (%): 44.18 -> 46.28Improved
Select gradient-norm-heavy samples (MaxGrad) instead of randomonline batch selection · MMLU training throughputsamples per second: Qwen-2.5-7B: 0.31 vs 2.27 full-datasetHurt
Select max-loss samples (MaxLoss) instead of randomonline batch selection · MMLUaverage accuracy (%): 54.26 -> 54.51No clear effect
Increase selected samples K per batch beyond 4 (ablation)UDS · MMLU (Llama-3.1-8B)average accuracy (%)Hurt
Use UDS at K=4 instead of full-dataset fine-tuningLlama-3.1-8B · MMLUaverage accuracy (%)Improved

Earlier methods this paper compares with

Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.

Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy