Research map / Large language models / Instruction and fine-tuning
Navigating the Pareto Frontier of Alignment: Spectrum-Adaptive Fine-Tuning for LLMs
Yaoyou Fan, Chao Zhang, Xiaoyu Tan, Chenxing Sun, Yu Yuan, Haoyu Feng, Lu Pan, Ke Zeng, Xunliang Cai
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use SFT instead of base model under 50% label noise | Qwen2.5-Math-1.5B · Math avg, 50% noise | avg accuracy: 11.0 -> 5.32 | Hurt |
| replace SFT with DFT under 50% label noise | Qwen2.5-Math-1.5B · Math avg, 50% noise | avg accuracy: 5.32 -> 24.01 | Improved |
| use harmonic interpolation instead of SFT under 50% noise | Qwen2.5-Math-1.5B · Math avg, 50% noise | avg accuracy: 5.32 -> 23.41 | Improved |
| use geometric instead of linear interpolation under 50% noise (ablation) | SAFT · Math avg, 50% noise | avg accuracy: 8.35 -> 20.07 | Improved |
| increase Geo-SAFT alpha from 0.5 to 2.0 under 50% noise (ablation) | Geo-SAFT · Math avg, 50% noise | avg accuracy: 20.07 -> 24.11 | Improved |
| replace SFT with Har-SAFT | Qwen2.5-Math-7B · Math avg (6 benchmarks) | avg accuracy: 20.46 -> 31.37 | Improved |
| replace DFT with Har-SAFT | Qwen2.5-Math-7B · Math avg (6 benchmarks) | avg accuracy: 30.20 -> 31.37 | Improved |
| replace DFT with Har-SAFT | Qwen3-14B · Math avg (6 benchmarks) | avg accuracy: 31.06 -> 33.50 | Improved |
| replace DFT with Har-SAFT | Qwen2.5-Math-1.5B · Math avg (6 benchmarks) | avg accuracy: 26.78 -> 27.48 | No clear effect |
| replace DFT with Geo-SAFT | Qwen3-8B · Math avg (6 benchmarks) | avg accuracy: 28.82 -> 29.59 | Mixed |
| use geometric instead of harmonic interpolation (ablation) | SAFT · Math avg, Qwen2.5-Math-7B | avg accuracy: 31.37 -> 29.99 | Hurt |
| replace DFT with Har-SAFT | Qwen2.5-Math-7B · Out-of-domain avg | avg accuracy: 27.28 -> 44.66 | Improved |
Closest papers by meaning
- Linearization Explains Fine-Tuning in Large Language Models (NeurIPS 2025 · similarity 0.89)
- Alignment Dynamics in LLM Fine-Tuning (NeurIPS 2026 · similarity 0.89)
- Data Efficient Adaptation in Large Language Models via Continuous Low-Rank Fine-Tuning (NeurIPS 2025 · similarity 0.89)
- Beyond Imitation: Leveraging Fine-grained Quality Signals for Alignment (ICLR 2024 · similarity 0.89)
- The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learning (ICLR 2024 · similarity 0.88)
- Get more for less: Principled Data Selection for Warming Up Fine-Tuning in LLMs (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
