Research map / Probabilistic and scientific modeling / Variational and generative sampling
Principled Gradient-Based MCMC for Conditional Sampling of Text
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes faithful gradient-based MCMC samplers (p-NCG Langevin-style, Gibbs-with-Langevin, and a hybrid) for sampling text from energy-based LMs with soft constraints, and shows prior gradient-based text samplers (COLD, MUCOLA, SVS) are unfaithful.
Problem
Sampling discrete text from energy-based models defined by pretrained language models with soft constraints; existing gradient-based MCMC text samplers do not converge to the intended target distribution.
Approach
Construct discrete gradient-informed proposals (ℓp-norm constrained gradient proposal, and a Gibbs-style single-position variant) whose transition probabilities are computable in closed form so they can be corrected with Metropolis-Hastings, yielding samplers with the correct stationary distribution; also a hybrid that switches from p-NCG to GwL.
Key results
- On sentiment control, p-NCG + GwL reaches 0.96 success with perplexity 23.61 vs. SVS 0.84 / 32.73 and MUCOLA 0.66 / 85.74.
- On topic control, p-NCG + GwL achieves 0.99 success and perplexity 5.17, versus SVS 0.92 / 13.9 and FUDGE 0.30 / 5.59.
- On position-constrained generation, p-NCG + GwL achieves the lowest perplexity among samplers (55.06 vs. 95.38 for MUCOLA and 79.13 for SVS-LANGEVIN).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use p-NCG instead of SVS | GPT-2-small fine-tuned on E2E · Topic control | Success: 0.92 ± 0.05 -> 0.96 ± 0.03 | Helped |
| Use p-NCG instead of SVS | GPT-2-small fine-tuned on E2E · Topic control | PPL: 13.9 ± 2.04 -> 6.82 ± 0.47 | Helped |
| Add GwL after p-NCG (ablation) | p-NCG · Topic control | PPL: 6.82 ± 0.47 -> 5.17 ± 0.38 | Helped |
| Add GwL after p-NCG (ablation) | p-NCG · Topic control | Success: 0.96 ± 0.03 -> 0.99 ± 0.02 | Helped |
| Use p-NCG instead of SVS | GPT-2-large · Sentiment control | Success: 0.84 ± 0.06 -> 0.92 ± 0.05 | Helped |
| Use p-NCG instead of SVS | GPT-2-large · Sentiment control | PPL: 32.73 ± 4.09 -> 39.03 ± 5.67 | Hurt |
| Add GwL after p-NCG (ablation) | p-NCG · Sentiment control | PPL: 39.03 ± 5.67 -> 23.61 ± 2.09 | Helped |
| Add GwL after p-NCG (ablation) | p-NCG · Sentiment control | Success: 0.92 ± 0.05 -> 0.96 ± 0.04 | Helped |
| Add GwL after p-NCG (ablation) | p-NCG · Position-constrained generation (COLLIE) | PPL: 71.46 ± 17.41 -> 55.06 ± 9.53 | Helped |
| Use lower p values for the p-norm constraint (tested from 1.0 to 2.0) (ablation) | p-NCG · Unconditional sampling from GPT-2-large | performance | Helped |
| Use random scan instead of systematic scan (ablation) | GwL · Text generation experiments | performance | Helped |
Surprising findings
- MUCOLA can approximate the true toy-model distribution fairly well for certain step sizes, yet still never converges exactly to it.
- Prior gradient-based text samplers (COLD, MUCOLA) are shown to be biased in their stationary distributions even though they perform well on downstream tasks.
Limitations
- Fixed sequence length N, so the sampler cannot change length or do insertions/deletions.
- Step size must be manually tuned per model via grid search.
- Mixing time grows exponentially as step size decreases, so unadjusted small-step proposals are impractical.
Experimental setup
Datasets: E2E SST2 COLLIE (Gutenberg subset, filtered to 43 constraint examples) Project Gutenberg
Benchmarks: E2E topic control SST2 sentiment control COLLIE positional constraints unconditional GPT-2 sampling
Metrics: success rate perplexity Distinct-n total variation distance energy
Baselines: COLD MUCOLA SVS SVS-LANGEVIN FUDGE GPT2
Builds on: MUCOLA COLD SVS Norm Constrained Gradient sampler D-MALA
Closest papers by meaning
- Controlled LLM Decoding via Discrete Auto-regressive Biasing (ICLR 2025 · similarity 0.87)
- Fast Sampling via Discrete Non-Markov Diffusion Models with Predetermined Transition Time (NeurIPS 2024 · similarity 0.87)
- Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputs (ICLR 2025 · similarity 0.86)
- Closing the Curious Case of Neural Text Degeneration (ICLR 2024 · similarity 0.85)
- Energy-Based Diffusion Language Models for Text Generation (ICLR 2025 · similarity 0.85)
- Constrained Sampling for Language Models Should Be Easy: An MCMC Perspective (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.