Research map / Large language models / Language, safety and interpretability

Guaranteed Generation from Large Language Models

ICLR 2025 · Large language models › Language, safety and interpretability · approach: Parallel and early-exit decoding · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes GUARD, which trains an autoregressive proposal to approximate the gold filtered distribution g(y)∝a(y)b(y) (via DPG, warm-started with constraint-aware prompting) and then applies rejection sampling so outputs strictly satisfy hard constraints.

Problem

Guaranteeing that all LLM outputs strictly satisfy hard constraints while distorting the base model's distribution as little as possible and keeping inference cost low; autoregressive models cannot in general exactly represent the constrained target distribution.

Approach

Define the gold filtered model g ∝ a·b as the ideal target (the I-projection of the base LLM onto constraint-satisfying distributions), show an autoregressive model cannot exactly match it in general, then approximate it with an autoregressive proposal a' trained by distributional policy gradient (warm-started from constraint-aware prompting) and enforce the constraint by rejection sampling from a

Key results

• On the lexical constraint task, GUARD raises the acceptance rate from 0.0023 to 0.416 (~180x) over the base Gemma-2B while keeping KL(g||g') low.
• On sentiment reversal, GUARD improves acceptance rate from 0.005 to 0.306 (~60x) relative to GPT-2 base.
• Warm-start DPG strictly Pareto-dominates SFT points in the acceptance-rate vs KL trade-off under a fixed 800K sampling budget.

Measured effects reported in the paper

Surprising findings

• Constraint-aware prompting alone boosts acceptance rate without training but yields distributions that greatly diverge from the gold model and show reduced diversity and positional bias.
• Under P≠NP, there exist polynomial-time autoregressive models and predicates such that no polynomial-time autoregressive model matches the gold filtered distribution.

Limitations

• Guarantees only hold relative to the filter b; if b fails to capture true requirements, undesirable outputs may still be produced.
• Experiments are limited to two simple scenarios (lexical constraints and positive ending sentiment) with small models (Gemma-2B, GPT-2).
• Downstream task performance of the guaranteed generator is not evaluated.

Experimental setup

Datasets: ROCStories

Benchmarks: lexical constraint 'amazing' generation (Gemma-2B) ROCStories positive ending sentiment reversal (GPT-2)

Metrics: KL(g||g') acceptance rate Self-BLEU semantic similarity UMAP visualization

Baselines: rejection sampling from base LLM SFT constraint-aware prompting Independent Metropolis-Hastings Quasi-Rejection Sampling

Builds on: Distributional Policy Gradient I-projection / information geometry rejection sampling Independent Metropolis-Hastings NADO

Closest papers by meaning

• Syntactic and Semantic Control of Large Language Models via Sequential Monte Carlo (ICLR 2025 · similarity 0.89)
• Controllable Generation via Locally Constrained Resampling (ICLR 2025 · similarity 0.88)
• Breaking AR’s Sampling Bottleneck: Provable Acceleration via Diffusion Language Models (NeurIPS 2025 · similarity 0.88)
• On the Query Complexity of Verifier-Assisted Language Generation (ICML 2025 · similarity 0.87)
• Scaling up Masked Diffusion Models on Text (ICLR 2025 · similarity 0.87)
• Reflection-Window Decoding: Text Generation with Selective Refinement (ICML 2025 · similarity 0.87)