Research map / Label-efficient and robust learning / Recognition and long-tail
Train on Validation (ToV): Fast data selection with applications to fine-tuning
Ayush Jain, Andrea Montanari, Eren Sasoglu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| select training data with ToV scores instead of random selection or max-uncertainty | Meta-Llama-3-8B instruction tuning · Slim Orca / Alpaca GPT-4 targets, n = 8x1024 | test log-loss | Improved |
| use ToV instead of LESS | Meta-Llama-3-8B instruction tuning · instruction tuning Exp 1-5 | test log-loss | Mixed |
| apply ToV when train and validation both come from Slim Orca | Meta-Llama-3-8B instruction tuning · Exp 3 (target = pool = Slim Orca) | test log-loss | Improved |
| good data selection compared with doubling the selected set size n | instruction tuning · test log-loss vs n | test log-loss | Improved |
| use ToV on NER versus random selection | xlm-roberta-base NER · Multinerd / Ai4p targets, n = 8x1024 | test log-loss | Improved |
| use max-uncertainty selection for NER | xlm-roberta-base NER · NER Exp 1-6 | test log-loss | Mixed |
| use LESS for NER selection | xlm-roberta-base NER · NER Exp 1-6 | test log-loss | No clear effect |
| Score-only versus Score+Random selection for instruction tuning (ablation) | ToV · instruction tuning Exp 1-5 | test log-loss | Mixed |
| Score-only versus Score+Random selection for NER (ablation) | ToV · NER | test log-loss | Mixed |
| use Method B (training resumed from validation-trained model) instead of Method A for scoring (ablation) | ToV on instruction tuning · Meta-Llama-3-8B | test log-loss | Hurt |
| use Method B instead of Method A for scoring (ablation) | ToV on NER · xlm-roberta-base | test log-loss | No clear effect |
Closest papers by meaning
- TSDS: Data Selection for Task-Specific Model Finetuning (NeurIPS 2024 · similarity 0.87)
- Taming Hyperparameter Sensitivity in Data Attribution: Practical Selection Without Costly Retraining (NeurIPS 2025 · similarity 0.87)
- Efficient Data Selection at Scale via Influence Distillation (NeurIPS 2025 · similarity 0.86)
- Data Mixing Optimization for Supervised Fine-Tuning of Large Language Models (ICML 2025 · similarity 0.86)
- Quick-Tune: Quickly Learning Which Pretrained Model to Finetune and How (ICLR 2024 · similarity 0.86)
- Diversified Batch Selection for Training Acceleration (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
