Research map / Large language models / Instruction and fine-tuning

Two-stage LLM Fine-tuning with Less Specialization and More Generalization

ICLR 2024 · Large language models › Instruction and fine-tuning · approach: Fine-tuning generalization and delta compression · 8 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ProMoT, a two-stage fine-tuning method (soft prompt tuning on the target task, then freezing the prompt and fine-tuning the model) that reduces format specialization and preserves or improves in-context learning on unseen tasks.

Problem

Fine-tuning LLMs on a single task causes format specialization, which erodes general in-context learning performance on unseen tasks.

Approach

Offload task-specific format learning into a removable soft prompt trained first with the model frozen, then fine-tune the model with that prompt kept attached, so format is absorbed by external parameters and the model's weights focus on semantics; the prompt is dropped at inference when the target format differs.

Key results

• On mT5 XXL fine-tuned on RTE, ProMoT + 1-shot raises the normalized average over 8 in-context tasks by +4.74 (1-shot) relative to pretrained, while standard fine-tuning drops it by -2.10.
• On mT5 XXL fine-tuned on WMT14 En-Fr, ProMoT reaches 41.30 BLEU vs 41.80 for standard fine-tuning, while lifting the normalized average by +1.35 vs standard fine-tuning's -8.37.
• ProMoT with multi-task training yields a normalized average gain of +8.35 (1-shot) vs +2.54 for standard multi-task fine-tuning.

Measured effects reported in the paper

Surprising findings

• Fine-tuning on NLI (RTE) improves few-shot summarization over the pretrained model under ProMoT, indicating cross-format generalization.
• Fine-tuning on En-Fr translation with ProMoT substantially improves En-De and En-Ro translation few-shot performance over the pretrained model.

Limitations

• No theoretical guarantee on how much format specialization the soft prompt absorbs.
• Experiments limited to models under 15B parameters due to compute.
• ProMoT does not completely eliminate specialization and can still underperform the pretrained model on some unseen tasks.

Experimental setup

Datasets: RTE WMT14 En-Fr SNLI OpenbookQA CB WiC TriviaQA web_questions WMT16 En-De WMT16 En-Ro XSum WikiLingua

Benchmarks: RTE WMT14 En-Fr SNLI OpenbookQA CB WiC TriviaQA web_questions WMT16 En-De WMT16 En-Ro XSum WikiLingua

Metrics: accuracy exact match BLEU Rouge-2 normalized average

Baselines: standard fine-tuning prompt tuning multi-task fine-tuning joint fine-tuning LoRA-based ProMoT pretrained model

Builds on: prompt tuning LoRA multi-task fine-tuning

Closest papers by meaning

• When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method (ICLR 2024 · similarity 0.90)
• DELIFT: Data Efficient Language model Instruction Fine-Tuning (ICLR 2025 · similarity 0.90)
• Data Efficient Adaptation in Large Language Models via Continuous Low-Rank Fine-Tuning (NeurIPS 2025 · similarity 0.90)
• An Emulator for Fine-tuning Large Language Models using Small Language Models (ICLR 2024 · similarity 0.90)
• LensLLM: Unveiling Fine-Tuning Dynamics for LLM Selection (ICML 2025 · similarity 0.90)
• Get more for less: Principled Data Selection for Warming Up Fine-Tuning in LLMs (ICLR 2024 · similarity 0.89)