Research map / Large language models / Reasoning and chain of thought

LARFT: Closing the Cognition-Action Gap for Length Instruction Following in Large Language Models

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Test-time compute scaling · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes LARFT, which combines length-awareness relabeling SFT with length-oriented RL under a unified optimization objective to improve LLM length instruction following while preserving general abilities.

Problem

LLMs struggle to precisely control output length; prior methods impose external length signals or objectives but ignore the model's intrinsic deficit in length cognition (awareness of length).

Approach

Diagnoses a "cognition-action gap": explicit length-awareness training alone does not improve length following, but initializing RL from a length-aware model helps. LARFT pairs SFT for injecting expert length knowledge with RL as the final optimization objective, unified via a coefficient lambda with cosine annealing scheduling, and uses relabeling to build length-awareness tasks.

Key results

• LARFT achieves the lowest LIFEBench LD and highest LS across all four models, improving LS by up to 12.30 points over the second-best method (Qwen2.5-3B-Instruct).
• On Lenctrl-Bench, LARFT reduces MAE to 7.72 (Qwen2.5-3B) and 7.00 (Qwen2.5-7B), versus 84.9 and 53.0 for PositionID, while SFT-based methods degrade sharply on short-length following.
• On IFEval, pure RL drops from 85.85 to 73.38 on Llama-3.1-8B, whereas LARFT keeps general instruction following at 81.35 on that model.

Surprising findings

• Length-awareness SFT alone yields no significant gain in length following, but initializing RL from it improves over standard RL by 3.93 points on average.
• SFT-based methods, despite improving long-generation quality, severely regress on short length constraints (MAE increases over 15 points).

Limitations

• Results are reported only on four Qwen and Llama instruct models of 3B to 8B parameters.
• Lambda sensitivity and cosine annealing ablations are run only on Qwen2.5-7B.

Experimental setup

Datasets: LIFEBench LongBench Lenctrl-Bench

Benchmarks: LIFEBench LongBench Lenctrl-Bench MMLU GSM8K IFEval GPQA

Metrics: length deviation length score MAE ROUGE-L accuracy

Baselines: SFT RL SFT+RL PositionID Ruler

Builds on: GRPO

Closest papers by meaning

• When More is Less: Understanding Chain-of-Thought Length in LLMs (ICLR 2026 · similarity 0.90)
• LIFEBENCH: Evaluating Length Instruction Following in Large Language Models (NeurIPS 2025 · similarity 0.90)
• Incentivizing Reasoning for Advanced Instruction-Following of Large Language Models (NeurIPS 2025 · similarity 0.88)
• The Illusion of Diminishing Returns: Measuring Long Horizon Execution in LLMs (ICLR 2026 · similarity 0.87)
• ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models (NeurIPS 2025 · similarity 0.87)
• Self-play with Execution Feedback: Improving Instruction-following Capabilities of Large Language Models (ICLR 2025 · similarity 0.87)