Research map / Large language models / Language, safety and interpretability
When Copying Is Hard: Copy-Constrained Decoding for Exact Span Reproduction
Jinghui Zhang, Lang Gao, Zongfang Liu, Ruihong Zeng, Zirui Song, Rui Yan, Kentaro Inui, Xiuying Chen
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Studies exact verbatim span reproduction from long context in LLMs and proposes a copy-constrained decoding approach to fix within-span drift and boundary errors.
Problem
LLMs often fail to reproduce a target span verbatim from context, drifting within the span or failing to stop at the correct boundary, which matters for tasks like embedding sensitive strings in generated code.
Approach
Constrains decoding to copy from the source context so that generated spans stay exact and terminate at the correct boundary; the abstract does not detail the mechanism.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add CopyGen plugin to frozen LLMs | Qwen3-4B, Qwen3-8B, Ministral-8B · Exact-copy benchmark, 3 backbones avg | EM: 51.9 -> 70.3 | Improved |
| add CopyGen on top of SFT backbone | LoRA SFT models · Exact-copy benchmark, 3 backbones avg | EM: 91.3 -> 95.2 | Improved |
| remove margin calibration (ablation) | CopyGen · Average of 4 domains, Qwen3-8B | EM: 76.97 -> 60.47 when removed | Improved |
| add adaptive hardness weighting in training (ablation) | CopyGen · Average of 4 domains, Qwen3-8B | EM: 63.67 -> 76.97 | Improved |
| add adaptive length proposal instead of fixed length (ablation) | CopyGen · Average of 4 domains, Qwen3-8B | Tokens/s: 354.94 -> 842.10 | Improved |
| add adaptive length proposal instead of fixed length (ablation) | CopyGen · Average of 4 domains, Qwen3-8B | EM: 74.32 -> 76.97 | Improved |
| replace learned bias head with fixed margin rules (ablation) | CopyGen · Qwen3-8B, 3 real domains avg | EM: 72.87 -> 59.45 (margin +1) | Hurt |
| reduce plugin training data to 10% (ablation) | CopyGen · Qwen3-8B, 4 domains avg | EM: 76.97 -> 70.88 | Hurt |
| reduce plugin training data to 50% (ablation) | CopyGen · Qwen3-8B, 4 domains avg | EM: 76.97 -> 76.69 | No clear effect |
| train on other two domains and test on held-out domain (ablation) | CopyGen · Qwen3-8B, 3 real domains avg | EM: 72.49 -> 65.53 (base 57.97) | Mixed |
| add rule-based constraints (PICARD-style, Longest-Copy) | frozen LLM · Exact-copy benchmark, 3 backbones avg | EM: 51.85 -> 51.61 (PICARD), 26.32 (Longest-Copy) | Hurt |
| integrate CopyGen in vLLM and SGLang | Qwen3-8B · 4 domains avg | EM: 60.69 -> 77.00 (vLLM) | Improved |
Closest papers by meaning
- LongGenBench: Benchmarking Long-Form Generation in Long Context LLMs (ICLR 2025 · similarity 0.87)
- Frontier Language Models Struggle to Copy: Text Can Be Better Viewed in 2D (NeurIPS 2026 · similarity 0.87)
- Repeat After Me: Transformers are Better than State Space Models at Copying (ICML 2024 · similarity 0.86)
- LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code (ICLR 2025 · similarity 0.86)
- Getting the most out of your tokenizer for pre-training and domain adaptation (ICML 2024 · similarity 0.85)
- Let the Code LLM Edit Itself When You Edit the Code (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
