Research map / Large language models / Instruction and fine-tuning
TuneAhead: Predicting Fine-tuning Performance Before Training Begins
Yuxiang Luo, Haonan Long, Chen Wang, Qiqi Duan, Xiaotian Lin, Yanwei Xu, Yuyu Luo, Weikai Yang, Nan Tang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add dynamic probe features to static-only features (ablation) | TuneAhead · MMLU performance prediction | RMSE (pp): 3.50 -> 1.47 | Improved |
| add static dataset features to dynamic-only features (ablation) | TuneAhead · MMLU performance prediction | RMSE (pp): 3.38 -> 1.47 | Improved |
| add static features to dynamic-only predictor on runs where only dynamic succeeded (ablation) | TuneAhead · MMLU (rescued-by-dynamic bucket) | Acc@2pp: 38.7 -> 75.0 | Improved |
| add dynamic features to static-only predictor on runs where only static succeeded (ablation) | TuneAhead · MMLU (rescued-by-static bucket) | Acc@2pp: 37.3 -> 100.0 | Improved |
| combine static and dynamic features when both subset predictors failed (ablation) | TuneAhead · MMLU (both-wrong bucket) | Acc@2pp: 12.5 -> 70.2 | Improved |
| add a 100-step probe (dynamic features) versus 0 steps (ablation) | TuneAhead · MMLU performance prediction | RMSE (pp): 3.50 -> 1.47 | Improved |
| extend probe from 100 to 200 steps (ablation) | TuneAhead · MMLU performance prediction | Acc@2pp: 82.5 -> 82.8 | No clear effect |
| extend probe from 100 to 200 steps (ablation) | TuneAhead · MMLU performance prediction | probe time (min): 7.76 -> 12.11 | Hurt |
| use TuneAhead instead of ProxyLM | TuneAhead · MMLU performance prediction | RMSE (pp): 2.11 -> 1.47 | Improved |
| use TuneAhead instead of ProxyLM on Llama-3-8B-Instruct | TuneAhead · MMLU (Llama-3-8B-Instruct) | RMSE (pp): 6.76 -> 5.02 | Improved |
| use TuneAhead instead of ProxyLM on TruthfulQA | TuneAhead · TruthfulQA MC2 performance prediction | RMSE (pp): 2.54 -> 2.17 | Improved |
| apply diagnosis-guided fixes (lower lr, SemDeDup, outlier removal) to a predicted-failure run | TuneAhead diagnosis · MMLU single run | MMLU accuracy: 20.2 -> 48.7 | Improved |
Closest papers by meaning
- DELIFT: Data Efficient Language model Instruction Fine-Tuning (ICLR 2025 · similarity 0.90)
- LensLLM: Unveiling Fine-Tuning Dynamics for LLM Selection (ICML 2025 · similarity 0.89)
- Get more for less: Principled Data Selection for Warming Up Fine-Tuning in LLMs (ICLR 2024 · similarity 0.89)
- Linearization Explains Fine-Tuning in Large Language Models (NeurIPS 2025 · similarity 0.89)
- Selecting Large Language Model to Fine-tune via Rectified Scaling Law (ICML 2024 · similarity 0.88)
- Data Mixing Optimization for Supervised Fine-Tuning of Large Language Models (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
