Research map / Large language models / Instruction and fine-tuning
Beyond Log Likelihood: Probability-Based Objectives for Supervised Fine-Tuning across the Model Capability Continuum
Gaotang Li, Ruizhong Qiu, Xiusi Chen, Heng Ji, Hanghang Tong
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace -log(p) with -p objective in model-strong math SFT | Qwen2.5-Math-1.5B SFT · Math avg (5 benchmarks) | accuracy: 17.00 -> 32.75 | Improved |
| exclude tokens with p<0.2 from -log(p) loss (ablation) | Qwen2.5-Math-7B SFT · Math avg (5 benchmarks) | accuracy: 22.67 -> 38.16 | Improved |
| replace -log(p) with -p on medical reasoning (model-intermediate) | SFT on m23k · Medical reasoning avg (10 benchmarks) | accuracy: LLaMA-3.1-8B 47.23 -> 45.89; Qwen2.5-Math-7B 33.56 -> 33.83 | No clear effect |
| replace -log(p) with -p on figfont puzzles (model-weak) | Qwen2.5-7B SFT · Figfont puzzles | Jaro-Winkler similarity: 82.48 -> 10.15 | Hurt |
| replace -log(p) with -p on low-resource multilingual instruction tuning | LLaMA-3.1-8B SFT · MMLU-ProX low-resource languages avg | accuracy: 16.1 -> 11.5 | Hurt |
| replace -log(p) with -p as backbone grows from 3B to 14B in general instruction tuning | Qwen2.5 SFT · AlpacaEval2 LC win rate of -p vs -log p | win rate: 41.0 -> 49.6 -> 57.5 | Mixed |
| replace -log(p) with -p for coding SFT | Qwen2.5-Coder-7B · EvalPlus avg | pass@1: 74.7 -> 77.8 | Improved |
| scale backbone from 3B to 32B in math SFT with -p versus NLL (ablation) | Qwen2.5 SFT · Math benchmarks | performance improvement of -p over NLL | Improved |
| train only on top-10% highest-probability tokens (instead of all tokens) (ablation) | Qwen2.5-Math-1.5B SFT, all objectives · Math avg | accuracy | Improved |
| include low-probability tokens in training (ablation) | Qwen2.5-Math-1.5B SFT · Math avg | accuracy | Hurt |
| increase alpha (more concave objective) in model-strong end (ablation) | Qwen2.5-Math-1.5B, f_alpha family · Math mean accuracy | accuracy | Improved |
| increase alpha toward 1 (more concave objective) in model-weak end (ablation) | Qwen2.5-7B, f_alpha family · Figfont Jaro-Winkler similarity | Jaro-Winkler similarity | Hurt |
Closest papers by meaning
- Model Fusion through Bayesian Optimization in Language Model Fine-Tuning (NeurIPS 2024 · similarity 0.90)
- Discriminative Finetuning of Generative Large Language Models without Reward Models and Human Preference Data (ICML 2025 · similarity 0.89)
- Preference-Oriented Supervised Fine-Tuning: Favoring Target Model over Aligned Large Language Models (AAAI 2025 · similarity 0.89)
- DELIFT: Data Efficient Language model Instruction Fine-Tuning (ICLR 2025 · similarity 0.89)
- Get more for less: Principled Data Selection for Warming Up Fine-Tuning in LLMs (ICLR 2024 · similarity 0.89)
- FisherSFT: Data-Efficient Supervised Fine-Tuning of Language Models Using Information Gain (ICML 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
