Research map / Large language models
Instruction and fine-tuning: research map
888 papers on Instruction and fine-tuning in Large language models, from ICML, NeurIPS, ICLR, CVPR, AAAI, ICCV and ECCV, and the journals TPAMI, IJCV and JMLR (2016–2026), grouped into 2 clusters and 8 approaches. The busiest year so far is 2026.
What it is
A pretrained language model becomes a useful assistant through fine-tuning on instructions and answers. This topic covers which data to use, cheap fine-tuning methods such as LoRA, and side effects such as lost safety or calibration after fine-tuning.
Terms you will see on this page:
- Instruction tuning
- fine-tuning on (instruction, response) pairs so the model follows requests.
- Parameter-efficient fine-tuning
- updating only a small set of added parameters, such as LoRA.
- Data selection
- choosing which training examples actually help.
Start reading with the most cited papers here:
- Language Models are Few-Shot Learners (NeurIPS 2020 · 2,966 citations)
- LISA: Reasoning Segmentation via Large Language Model (CVPR 2024 · 381 citations)
Within Large language models, its share held steady from 10.5% in 2023–24 to 9.5% in 2025–26 (208 → 653 papers at ICML, NeurIPS, ICLR, CVPR and AAAI, the venues with data for all four years).
Explore Instruction and fine-tuning in the interactive map
Working on something in this topic? Describe your idea in scime atlas to see which approach it falls under, the closest papers by meaning and how crowded the spot has become.
What changed recently
Approaches whose share of the topic moved most between 2023–24 and 2025–26 (ICML, NeurIPS, ICLR, CVPR and AAAI):
- Fine-tuning for reasoning data rising: 12% → 20% of the topic (25 → 131 papers)
- Safety alignment after fine-tuning rising: 9% → 16% of the topic (18 → 102 papers)
- Fine-tuning generalization and delta compression rising: 17% → 19% of the topic (35 → 124 papers)
- Calibration and knowledge in LLM tuning falling: 16% → 8% of the topic (34 → 50 papers)
- Instruction data selection falling: 11% → 6% of the topic (22 → 40 papers)
Papers from 2025–2026 that are already cited most:
- SWIFT: A Scalable Lightweight Infrastructure for Fine-Tuning (AAAI 2025 · 40 citations)
- CXPMRG-Bench: Pre-training and Benchmarking for X-ray Medical Report Generation on CheXpert Plus Dataset (CVPR 2025 · 20 citations)
- Scaling Trends for Data Poisoning in LLMs (AAAI 2025 · 7 citations)
Approaches and key papers
Instruction tuning · 715 papers
Keywords: instruction tuning, instructions, instruction following
Approaches in this cluster:
- Efficient LLM fine-tuning systems (224 papers)
Cut memory and compute of fine-tuning through sparsity, projections, and distributed training. - Fine-tuning generalization and delta compression (161 papers)
Study fine-tune deltas, emulation, and mixtures to preserve generality when specializing LLMs. - Fine-tuning for reasoning data (161 papers)
Improve reasoning via tailored losses, scaled instruction data, and efficient distillation. - Safety alignment after fine-tuning (125 papers)
Defend against harmful fine-tuning with post-hoc and in-training safety alignment. - Low-rank adaptation variants (44 papers)
Improve LoRA with adaptive rank, scaling, contextual, and multi-adapter designs.
Most cited and most cited since 2024:
- Language Models are Few-Shot Learners (NeurIPS 2020 · 2,966 citations)
- Zhongjing: Enhancing the Chinese Medical Capabilities of Large Language Model through Expert Feedback and Real-World Multi-Turn Dialogue (AAAI 2024 · 110 citations)
- WizardLM: Empowering Large Pre-Trained Language Models to Follow Complex Instructions (ICLR 2024 · 105 citations)
- Cell2Sentence: Teaching Large Language Models the Language of Biology (ICML 2024 · 62 citations)
Efficient fine-tuning and adaptation · 173 papers
Keywords: fine tuning, lora, adaptation
Approaches in this cluster:
- Calibration and knowledge in LLM tuning (86 papers)
Fine-tune LLMs for calibration and multilingual ability using gradient-based parameter selection. - Instruction data selection (63 papers)
Select and structure instruction-tuning data and evaluate robustness of instruction-tuned models. - Gradient-free adaptation and merging (24 papers)
Adapt LLMs through hypernetworks, model merging, and search without full backpropagation.
Most cited and most cited since 2024:
- LISA: Reasoning Segmentation via Large Language Model (CVPR 2024 · 381 citations)
- The Flan Collection: Designing Data and Methods for Effective Instruction Tuning (ICML 2023 · 113 citations)
- WizardCoder: Empowering Code Large Language Models with Evol-Instruct (ICLR 2024 · 81 citations)
- LayoutLLM: Layout Instruction Tuning with Large Language Models for Document Understanding (CVPR 2024 · 60 citations)
Related topics in Large language models
- Reasoning and chain of thought (1,574)
- Retrieval and knowledge (1,084)
- Agents, code and math (1,748)
- Alignment and preferences (672)
- AI and society (913)
- Language, safety and interpretability (3,002)
