Research map / Large language models / Instruction and fine-tuning
Fine-tuning for reasoning data
Improve reasoning via tailored losses, scaled instruction data, and efficient distillation.
158 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2019–2026) · an approach in Instruction and fine-tuning, keyword cluster “instruction tuning · instructions · instruction following”
State History Benchmarks Tried Checklist Preprints Who Export
Where it stands
Still moving The best reported result is still improving on 1 of 1 tracked benchmark.
25 papers in 2023–24 and 131 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic grew, from 12.0% to 20.1%.
Open questions the data points to
- GSM8K (accuracy): best went from 84.7 (up to 2024) to 95.5 in 2025 — The First Few Tokens Are All You Need: An Efficient and Effective Unsupervised Prefix Fine-Tuning Method for Reasoning Models (NeurIPS 2025 · 1 citation)
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2019 · Predicting Concrete and Abstract Entities in Modern Poetry (AAAI 2019 · 1 citation)
- 2021 · TFix: Learning to Fix Coding Errors with a Text-to-Text Transformer (ICML 2021)
- 2023 · How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources (NeurIPS 2023 · 38 citations)
- 2024 · Cell2Sentence: Teaching Large Language Models the Language of Biology (ICML 2024 · 62 citations)
- 2025 · VideoRFT: Incentivizing Video Reasoning Capability in MLLMs via Reinforced Fine-Tuning (NeurIPS 2025 · 6 citations)
- 2026 · STAR-1: Safer Alignment of Reasoning LLMs with 1K Data (AAAI 2026 · 3 citations)
Earliest papers here
- Predicting Concrete and Abstract Entities in Modern Poetry (AAAI 2019 · 1 citation)
- TFix: Learning to Fix Coding Errors with a Text-to-Text Transformer (ICML 2021)
- How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources (NeurIPS 2023 · 38 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
GSM8K · accuracy (higher is better)
4 papers, 2024–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2024 | 84.7 | MAmmoTH2: Scaling Instructions from the Web (NeurIPS 2024 · 2 citations) |
| 2025 | 95.5 | The First Few Tokens Are All You Need: An Efficient and Effective Unsupervised Prefix Fine-Tuning Method for Reasoning Models (NeurIPS 2025 · 1 citation) |
What has been tried
56 papers here report 666 measured changes (a component added, removed or swapped, compared with a baseline): helped 73%, no clear effect 8%, hurt 17%, mixed 2%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Add natural-language pretraining before Cell2Sentence training | Immune tissue, 17 cell types · R2 | 0.873 -> 0.949 | Helped | Cell2Sentence: Teaching Large Language Models the Language of Biology (ICML 2024 · 62 citations) |
| Lower sampling temperature from τ=1.0 to τ=0.7 | MP-20 · Metastable (M3GNet) | 35.4% -> 49.8% | Helped | Fine-Tuned Language Models Generate Stable Inorganic Materials as Text (ICLR 2024 · 35 citations) |
| Add coarse-grained reward weighting to OpenChat | AlpacaEval · win rate (%) | 89.1 -> 89.5 | Helped | OpenChat: Advancing Open-source Language Models with Mixed-Quality Data (ICLR 2024 · 27 citations) |
| Use self-distillation training with the learner's own sampled candidates | GSM8K · m1@t5 | 39.0 -> 39.5 | Helped | Recursive Introspection: Teaching Language Model Agents How to Self-Improve (NeurIPS 2024 · 15 citations) |
| Add cross-modal CoT refinement to the training data | VSI-Bench · accuracy | -2.3% | Helped | VideoRFT: Incentivizing Video Reasoning Capability in MLLMs via Reinforced Fine-Tuning (NeurIPS 2025 · 6 citations) |
| Increase LoRA matrix rank | Four binary classification GLUE datasets · Pearson correlation with Exact | Hurt | DataInf: Efficiently Estimating Data Influence in LoRA-tuned LLMs and Diffusion Models (ICLR 2024 · 5 citations) | |
| Train with MPU and GPU data rather than CPU-only data | GCC ARC (MPU) Code Generation · BLEU-4 | 19.29 -> 19.94 | Helped | ComBack: A Versatile Dataset for Enhancing Compiler Backend Development Efficiency (NeurIPS 2024 · 5 citations) |
| use update-gradient tensor importance instead of update magnitude | SciTLDR · ROUGE-1 | 32.7 -> 33.1 | Helped | Towards Green AI in Fine-tuning Large Language Models via Adaptive Backpropagation (ICLR 2024 · 4 citations) |
| Increase label scale to γ=3.0 | Continuous weight distribution · loss | Hurt | Understanding Catastrophic Forgetting in Language Models via Implicit Inference (ICLR 2024 · 4 citations) | |
| Convert fine-tuned Llama-3-8B to 4-bit WebLLM | All domains · HGood (%) | 77 -> 64 | Hurt | Hints-In-Browser: Benchmarking Language Models for Programming Feedback Generation (NeurIPS 2024 · 3 citations) |
| add difficulty-aware augmentation to GRPO | SEED-Bench-R1 L1 · score | 39.6 -> 41.7 (+2.1) | Helped | DeepVideo-R1: Video Reinforcement Fine-Tuning via Difficulty-aware Regressive GRPO (NeurIPS 2025 · 2 citations) |
| Fine-tune LLaMA3-70B with LoRA rather than use the base model | OpenDebateEvidence · ROUGE-1 F1 (%) | 33.8 ± 1.2 -> 37.2 ± 1.0 | Helped | OpenDebateEvidence: A Massive-Scale Argument Mining and Summarization Dataset (NeurIPS 2024 · 2 citations) |
What reviewers will expect
Datasets and baselines named in the 63 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- GSM8K · 11 of 63 papers
- LIMO · 3 of 63 papers
- MATH500 · 3 of 63 papers
- AIME 2024 · 3 of 63 papers
- SST-2 · 2 of 63 papers
- Alpaca · 2 of 63 papers
- MinervaMATH · 2 of 63 papers
- AIME 2025 · 2 of 63 papers
Baselines
- SFT · 6 of 63 papers
- GPT-4o · 4 of 63 papers
- GRPO · 3 of 63 papers
- GPT-4 · 2 of 63 papers
- Llama 2 7B Chat · 2 of 63 papers
- Llama 3 8B Instruct · 2 of 63 papers
- DeepSeek-R1 · 2 of 63 papers
- DPO · 2 of 63 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 156 papers; from OpenAlex and the papers' affiliations).
- Google · 8 papers
- Carnegie Mellon University · 7 papers
- Meta · 4 papers
- Tencent · 4 papers
- Stanford University · 4 papers
- Google DeepMind · 4 papers
- Peking University · 4 papers
- Chinese Academy of Sciences · 3 papers
- National University of Singapore · 3 papers
- Zhejiang University · 3 papers
Use it
Download BibTeX of 24 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Efficient LLM fine-tuning systems (219 papers)
- Fine-tuning generalization and delta compression (159 papers)
- Safety alignment after fine-tuning (120 papers)
- Low-rank adaptation variants (43 papers)
Elsewhere in the topic
- Calibration and knowledge in LLM tuning (85 papers)
- Instruction data selection (63 papers)
- Gradient-free adaptation and merging (24 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).