Research map / Large language models / Language, safety and interpretability
Hansel: Output Length Controlling Framework for Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Hansel, which finetunes LLMs on data augmented with periodically inserted hidden special tokens that encode remaining output word count, giving precise and extrapolable output length control.
Problem
LLMs cannot precisely control output length; prior length-control methods need pretraining or architecture changes, and prompt-based finetuning generalizes poorly to target lengths unseen in training.
Approach
Augment finetuning targets with special tokens |x><y| inserted every Delta words that encode the remaining number of words, plus a residual delta with label masking to avoid abrupt termination; the method needs only finetuning and no architecture change.
Key results
- On Phi-2, Hansel reduces length MAE to 0.09 (DailyDialog), 0.05 (MultiWOZ), 0.26 (XSum), 0.39 (CNN/DM), a 45-75% reduction over Gretel while keeping ROUGE-L and G-Eval.
- At extrapolated target lengths (5 to 130 words), Hansel's MAE stays small (e.g., 2.07 at 130 words on DailyDialog vs 18.51 for Gretel).
- Infinite generation occurred 8.4 and 15.3 times per 10,000 generations for vanilla and Gretel, and zero times for Hansel.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use periodic hidden length tokens instead of prompt-only length finetuning | Phi-2 length control · DailyDialog, MultiWOZ, XSum, CNN/DM | MAE: DailyDialog: 0.24 -> 0.09; MultiWOZ: 0.11 -> 0.05; XSum: 0.47 -> 0.26; CNN/DM: 1.56 -> 0.39 | Helped |
| use periodic hidden length tokens instead of prompt-only length finetuning | Phi-2 length control · DailyDialog | ROUGE-L: 0.27 -> 0.33 | Helped |
| use periodic hidden length tokens instead of prompt-only length finetuning | Phi-2 length control · MultiWOZ | ROUGE-L: 0.34 -> 0.34 | No clear effect |
| use periodic hidden length tokens instead of prompt-only length finetuning | Phi-2 length control · XSum, CNN/DM | ROUGE-L: XSum: 0.33 -> 0.34; CNN/DM: 0.30 -> 0.31 | Helped |
| use periodic hidden length tokens instead of prompt-only length finetuning | Phi-2 length control · DailyDialog, MultiWOZ, XSum, CNN/DM | G-Eval: DailyDialog: 3.41 -> 3.55; MultiWOZ: 3.55 -> 3.54; XSum: 3.22 -> 3.26; CNN/DM: 3.82 -> 3.86 | Mixed |
| use periodic hidden length tokens instead of prompt-only length finetuning | Phi-2 length control · target length 130 words across four datasets | MAE: DailyDialog: 18.51 -> 2.07; MultiWOZ: 23.85 -> 7.47; XSum: 10.37 -> 1.02; CNN/DM: 7.38 -> 1.00 | Helped |
| set residual length δ to 0 rather than 1 (ablation) | Hansel · DailyDialog | MAE: 0.09 -> 0.08 | Helped |
| increase residual length δ (ablation) | Hansel · DailyDialog, MultiWOZ, XSum, CNN/DM | MAE: DailyDialog (Δ=20): 0.08, 0.09, 0.12, 0.16; MultiWOZ (Δ=20): 0.05, 0.05, 0.07, 0.09; XSum (Δ=20): 0.19, 0.26, 0.32, 0.38 | Hurt |
| use nonzero residual length δ to avoid ending immediately at the zero token (ablation) | Hansel · dialogue generation | abrupt termination | Helped |
| vary stride Δ and residual length δ (ablation) | Hansel · DailyDialog, MultiWOZ, XSum, CNN/DM | ROUGE-L: DailyDialog: 0.32–0.33; MultiWOZ: 0.34; XSum: 0.34; CNN/DM: 0.33–0.34 | No clear effect |
| train for more than 2 epochs (ablation) | Hansel · four datasets | MAE and ROUGE-L | Hurt |
| add 50 Hansel instruction samples for cross-task length control (ablation) | Phi-2 cross-task models · summarization and dialogue (40-word targets) | MAE: 13.53 to 2.62; 31.50 to 0.93 | Helped |
Surprising findings
- Prompt-based vanilla* models with the target length in the prompt but no length training break down in quality on dialogue datasets.
- Length control transfers across tasks and domains with only 50 instruction samples.
Limitations
- Evaluation limited to summarization and dialogue on four English datasets, with models of about 2.7-3B parameters, not large LLMs.
- Length control is measured in words; the paper notes difficulty varies by unit and it is not tested in other units beyond a sentence-word demo.
- Nonzero delta is needed to avoid abrupt termination, trading off length-control MAE (delta=0 gave lowest MAE but was rejected for quality reasons).
Experimental setup
Datasets: CNN/DM XSum DailyDialog MultiWOZ WMT19 de-en
Benchmarks: CNN/DM XSum DailyDialog MultiWOZ
Metrics: MAE of output length ROUGE-L ROUGE-1 ROUGE-2 G-Eval
Baselines: vanilla finetuning Gretel (prompt-based length finetuning)
Builds on: Chain-of-thought prompting AdamW Rotary positional embedding
Closest papers by meaning
- Long-Short Alignment for Effective Long-Context Modeling in LLMs (ICML 2025 · similarity 0.89)
- LongWriter: Unleashing 10,000+ Word Generation from Long Context LLMs (ICLR 2025 · similarity 0.89)
- InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memory (NeurIPS 2024 · similarity 0.88)
- Dataset Decomposition: Faster LLM Training with Variable Sequence Length Curriculum (NeurIPS 2024 · similarity 0.88)
- LIFEBENCH: Evaluating Length Instruction Following in Large Language Models (NeurIPS 2025 · similarity 0.88)
- Beyond Single-Task: Robust Multi-Task Length Generalization for LLMs (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.