Research map / Large language models / Instruction and fine-tuning
PriFT: Prior-Support Guided Token Reweighting for Supervised Fine-Tuning
Ke Wang, Shuangqi Li, Mathieu Salzmann, Pascal Frossard
AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PriFT, a token-reweighted SFT method that uses prior (pretrained-model) support to weight training tokens, aiming to improve SFT generalization toward RL-level performance.
Problem
SFT fits fixed demonstrations token by token, including targets poorly aligned with the pretrained model's distribution, leading to weaker generalization than RL.
Approach
Uses prior-support information from the pretrained model to reweight tokens during SFT, extending recent token-reweighted SFT methods that assign larger weights to tokens the model already favors.
Experimental setup
Baselines: standard SFT token-reweighted SFT methods reinforcement learning
Closest papers by meaning
- UFT: Unifying Supervised and Reinforcement Fine-Tuning (NeurIPS 2025 · similarity 0.89)
- Portable Reward Tuning: Towards Reusable Fine-Tuning across Different Pretrained Models (ICML 2025 · similarity 0.88)
- AceReason-Nemotron 1.1: Advancing Math and Code Reasoning through SFT and RL Synergy (ICLR 2026 · similarity 0.87)
- The Best Instruction-Tuning Data are Those That Fit (NeurIPS 2025 · similarity 0.87)
- Implicit Reward as the Bridge: A Unified View of SFT and DPO Connections (NeurIPS 2025 · similarity 0.87)
- Self-Improving Embodied Foundation Models (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
