Research map / Large language models / Language, safety and interpretability
The Hidden Cost of Structured Generation in LLMs: Draft-Conditioned Constrained Decoding
Avinash Reddy, Thayne T. Walker, Jaime S. Ide, Amrit Singh Bedi
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add draft conditioning before constrained decoding (DCCD vs CD) | Llama-3.2-1B constrained decoding · GSM8K | strict structured accuracy: 15.24 -> 39.04 | Improved |
| add draft conditioning (DCCD vs CD) | Qwen2.5-1.5B constrained decoding · MATH500 | strict structured accuracy: 15.00 -> 38.20 | Improved |
| add draft conditioning (DCCD vs CD) | Qwen2.5-14B constrained decoding · GSM8K | strict structured accuracy: 86.43 -> 95.15 | Improved |
| add draft conditioning (DCCD vs CD) | Llama-3.2-1B constrained decoding · GSM Symbolic | strict structured accuracy: 0.00 -> 9.00 | Improved |
| add draft conditioning (DCCD vs CD) | Qwen2.5-7B constrained decoding · FOLIO | strict structured accuracy: 19.70 -> 31.53 | Improved |
| average over all benchmarks, DCCD vs CD at 1B | Llama-3.2-1B · average of four benchmarks | strict accuracy: 10.2% -> 20.9% | Improved |
| add decode-time grammar masking on top of structural prompt (ablation) | Qwen2.5-1.5B · MATH500 | answer accuracy: 41.4 -> 15.0 | Hurt |
| add decode-time grammar masking on top of structural prompt (ablation) | Qwen2.5-14B · MATH500 | answer accuracy: 70.8 -> 47.6 | Hurt |
| mention the structural schema in the prompt (no CD) (ablation) | Qwen2.5-14B · MATH500 | answer accuracy: 76.0 -> 70.8 | Hurt |
| mention the structural schema in the prompt (no CD) (ablation) | Qwen2.5-7B · MATH500 | answer accuracy: 71.4 -> 71.2 | No clear effect |
| compose 7B draft model plus 1.5B constrainer vs single 14B CD | DCCD composition · GSM8K | accuracy per billion parameters: 6.1 -> 10.7 | Improved |
| compose 1.5B+1.5B DCCD vs 8B CD | DCCD composition · MATH500 | accuracy per billion parameters: 3.6 -> 12.7 | Improved |
Closest papers by meaning
- Multi-Branch Self-Drafting for LLM Inference Acceleration (AAAI 2025 · similarity 0.89)
- Grammar-Aligned Decoding (NeurIPS 2024 · similarity 0.89)
- Constrained Decoding for Diffusion Language Models via Efficient Inference over Finite Automata (NeurIPS 2026 · similarity 0.89)
- Guiding LLMs The Right Way: Fast, Non-Invasive Constrained Generation (ICML 2024 · similarity 0.89)
- Sample Smart, Not Hard: Correctness-First Decoding for Better Reasoning in LLMs (ICLR 2026 · similarity 0.89)
- Training-Free Loosely Speculative Decoding: Accepting Semantically Correct Drafts Beyond Exact Match (ICLR 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
