AI Paper Atlas

Research map / Large language models / Instruction and fine-tuning

Differential Fine-Tuning Large Language Models Towards Better Diverse Reasoning Abilities

Xiaosong Yuan, Chen Shen, Shaotian Yan, kaiyuan liu, Xiaofeng Zhang, Sinan Fan, Liang Xie, Wenxiao Wang, Renchu Guan, Ying Wang, Jieping Ye

ICLR 2026 · Large language models › Instruction and fine-tuning · approach: Fine-tuning generalization and delta compression

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add DiFT to Mistral-7B mix-up Math+Code SFTMix-Math-Code · Mistral-7B ATAATA: 52.87 -> 54.80Improved
add DiFT to continual Math-Code SFTContinual-Math-Code · Mistral-7B ATAATA: 64.58 -> 65.81Improved
add DiFT to continual Math-Logic SFT on Mistral-7BContinual-Math-Logic · Mistral-7B ATAATA: 16.94 -> 20.06Improved
add DiFT to Code+Logic mix-up on Llama3-8BMix-Code-Logic · Llama3-8B GSM8kaccuracy: 52.31 -> 41.09Hurt
add DiFT to Instruct model Logic+CSQA mix SFTLlama3-8B-Instruct · LogiQA2accuracy: 32.64 -> 34.48Improved
fine-tune only non-DSR parameters (inverse DiFT) in mix-up (ablation)DiFT · Mix-Math-Code GSM8kaccuracy: 61.26 vs 67.02Improved
fine-tune only DSR in continual SFT (inverse DiFT) (ablation)DiFT · Continual-Math-Code GSM8kaccuracy: 25.17 vs 46.32Improved
use top-DSR rows instead of random rows of same count, mix-up (ablation)DiFT · Mix-Math-Code GSM8kaccuracy: 62.35 -> 67.02Improved
use top-DSR rows instead of random rows, continual (ablation)DiFT · Continual-Math-Code xGLUEpass rate: 0.8192 -> 1.0557Improved
increase DSR rows from 100 to 200 (ablation)DiFT · GSM8kaccuracyHurt
transplant CSQA DSR into math-tuned model (ablation)math-sft · GSM8kaccuracy: 61.64 -> 41.02Hurt
transplant math DSR into CSQA-tuned model (ablation)csqa-sft · GSM8kaccuracy: 8.79 -> 22.21Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy