Research map / Large language models / Instruction and fine-tuning
Beyond Single-Task: Robust Multi-Task Length Generalization for LLMs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Meta-RFFT, multi-task rule-following fine-tuning on 74 of 86 constructed length-generalization tasks, enabling cross-task length generalization on unseen tasks with minimal fine-tuning or 1-shot prompting.
Problem
Length generalization in LLMs has been studied mainly for single symbolic tasks trained from scratch on small transformers; single-task RFFT requires per-task fine-tuning, so cross-task length generalization for practical LLMs is unaddressed.
Approach
Two-stage pipeline: first fine-tune an instruction-tuned LLM on a large multi-task rule-following dataset (RF-pretraining, 74 tasks, lengths 1-15), then adapt to unseen tasks with 1-5 length samples or a single rule-following exemplar in context.
Key results
- 32B Meta-RFFT fine-tuned on 1-5 digit addition achieves ~95% accuracy on 30-digit addition, versus DeepSeek-R1-671B 72% and QwQ-32B 32%.
- 32B Meta-RFFT improves ACC_Len30 by 14% over DeepSeek-R1-671B (84%) and 19% over QwQ-32B (79%) on downstream tasks.
- With 1-shot prompting, 32B RF-pretrained model reaches Max_Len_90% of 28.5, 14.5 above the base model and 10.3 above vanilla RFFT.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply Meta-RFFT before fine-tuning on 1-5-digit addition | Qwen2.5-32B-Instruct · 30-digit addition | accuracy: 95% vs. vanilla RFFT (40%) | Helped |
| Use Meta-RFFT instead of DeepSeek-R1-671B | 30-digit addition · 30-digit addition | accuracy: 95% vs. 72% | Helped |
| Use Meta-RFFT instead of QwQ-32B | 30-digit addition · 30-digit addition | accuracy: 95% vs. 32% | Helped |
| RF-pretrain on 74 diverse tasks rather than only 1 task (ablation) | Meta-RFFT · Downstream length-generalization tasks | accuracy | Helped |
| Use outcome-reward PPO instead of supervised rule-following fine-tuning (ablation) | Qwen2.5-7B · Length generalization | accuracy | Hurt |
| Pretrain on code-based rules, then adapt using natural-language rules (ablation) | Meta-RFFT · LC Add Digits, lengths 12-30 | accuracy | Helped |
| Add synthetic tasks to RF-pretraining for 1-shot rule-following (ablation) | Meta-RFFT · Downstream tasks, 1-shot | Max_Len_90%: 28.5, an improvement of 14.5 over the base model and even 10.3 over the vanilla RFFT model | Helped |
| RF-pretrain on multiple tasks before downstream adaptation instead of using vanilla RFFT alone | Qwen2.5-32B-Instruct · Downstream tasks | Max_Len_90% | Helped |
| Increase RF-pretraining data or training steps in the early phase (ablation) | Meta-RFFT · Downstream length-generalization tasks | accuracy | Helped |
| Increase RF-pretraining data or training steps after performance stabilizes (ablation) | Meta-RFFT · Downstream length-generalization tasks | accuracy | No clear effect |
| Increase downstream adaptation data or training steps in the early phase (ablation) | Meta-RFFT · Downstream length-generalization tasks | accuracy | Helped |
| Increase downstream adaptation data or training steps after performance stabilizes (ablation) | Meta-RFFT · Downstream length-generalization tasks | accuracy | No clear effect |
Surprising findings
- PPO with outcome reward led models to shortcut to direct answers, underperforming SFT-based rule following.
- Single-task RF-pretraining underperforms multi-task (74-task) RF-pretraining for length generalization.
Limitations
- Restricted to controlled synthetic tasks (code execution, number processing, symbolic/logical reasoning); does not address complex real-world long-horizon tasks.
- Defining 'length' as a complexity measure is hard outside controlled settings.
- Relies on manually written Python generator scripts per task, which is costly to scale.
Experimental setup
Datasets: LeetCode problems NUPA Big-Bench Hard Symbolic Reasoning (coin flip, last letter concatenation)
Benchmarks: LeetCode length generalization tasks (8 downstream) NUPA tasks (4 downstream)
Metrics: accuracy at length 30 (ACC_Len30) Max_Len_90% (max length with >=90% accuracy) accuracy by length
Baselines: direct answer fine-tuning scratchpad fine-tuning vanilla RFFT DeepSeek-R1-671B QwQ-32B DeepSeek-R1-Distill-7B Llama-3.1-Tulu-3.1-8B PPO reinforcement learning
Builds on: RFFT scratchpad fine-tuning PiSSA PPO
Closest papers by meaning
- The Role of Sparsity for Length Generalization in LLMs (ICML 2025 · similarity 0.90)
- Long-Short Alignment for Effective Long-Context Modeling in LLMs (ICML 2025 · similarity 0.90)
- LIFEBENCH: Evaluating Length Instruction Following in Large Language Models (NeurIPS 2025 · similarity 0.90)
- Two-stage LLM Fine-tuning with Less Specialization and More Generalization (ICLR 2024 · similarity 0.89)
- Scaling Instruction-tuned LLMs to Million-token Contexts via Hierarchical Synthetic Data Generation (ICLR 2025 · similarity 0.89)
- Extrapolation by Association: Length Generalization Transfer In Transformers (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.