Research map / Large language models / Instruction and fine-tuning
DRIFT: Difference-Aware Reinforcement Through Iterative Fine-Tuning for Language Model
Wenjie Liao, Xiaohui Song, Haonan Lu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add DRIFT iteration 1 after SFT epoch 1 | Qwen2.5-3B-Instruct · Summarization 5-task average | ROUGE-1: 44.00 -> 47.29 | Improved |
| add DRIFT iteration 1 after SFT epoch 1 | Llama-3.1-8B-Instruct · Summarization 5-task average | ROUGE-1: 45.15 -> 51.61 | Improved |
| add DRIFT iteration 1 after SFT epoch 1 | Llama-3.1-8B-Instruct · SAMSum | ROUGE-1: 49.93 -> 65.94 | Improved |
| add DRIFT iteration 1 after SFT epoch 1 | Qwen2.5-3B-Instruct · SAMSum | ROUGE-1: 47.47 -> 54.64 | Improved |
| train a second SFT epoch instead of DRIFT | Qwen2.5-3B-Instruct · Summarization 5-task average | ROUGE-1: 44.00 -> 43.71 | Hurt |
| train a second SFT epoch instead of DRIFT | Llama-3.1-8B-Instruct · Summarization 5-task average | ROUGE-2: 27.03 -> 25.50 | Hurt |
| run more DRIFT iterations (iteration 4 vs 1) | Llama-3.1-8B-Instruct · Summarization 5-task average | ROUGE-1: 51.61 -> 55.04 | Improved |
| use DRIFT instead of SPIN at iteration 1 | Qwen2.5-3B-Instruct · Summarization 5-task average | ROUGE-1: 46.03 -> 47.29 | Improved |
| add Difference-Aware Masking (DAM) (ablation) | DRIFT · Summarization 5-task average | ROUGE-1: 46.49 -> 47.29 | Improved |
| add Occurrence-Aware Loss (OAL) (ablation) | DRIFT · Summarization 5-task average | ROUGE-1: 46.99 -> 47.29 | Improved |
| update opponent each iteration instead of training more epochs on fixed data (ablation) | DRIFT · Summarization 5-task average | ROUGE-1: 47.45 -> 48.35 | Improved |
| add DRIFT iteration 1 after SFT epoch 1 | Qwen2.5-3B-Instruct · Open LLM Leaderboard average | accuracy: 61.47 -> 63.07 | Improved |
Closest papers by meaning
- Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models (ICML 2024 · similarity 0.90)
- IRIS: Interpolative Rényi Iterative Self-play for Large Language Model Fine-Tuning (NeurIPS 2026 · similarity 0.89)
- SPACE: Noise Contrastive Estimation Stabilizes Self-Play Fine-Tuning for Large Language Models (NeurIPS 2025 · similarity 0.88)
- Triplets Better Than Pairs: Towards Stable and Effective Self-Play Fine-Tuning for LLMs (NeurIPS 2025 · similarity 0.88)
- Training Language Models to Self-Correct via Reinforcement Learning (ICLR 2025 · similarity 0.88)
- Self-Adapting Language Models (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
