Research map / Image generation / Fast diffusion sampling
Show Me How To Revise: Improving Lexically Constrained Sentence Generation with XLNet
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Guide MCMC action and position selection with the learned classifier prior | L-MCMC · One-Billion-Word constrained generation, k=1 | NLL (GPT-2): 4.549 -> 3.762 | Helped |
| Use XLNet rather than LSTM as the MCMC language model | MCMC-C · One-Billion-Word constrained generation, k=1 | NLL (GPT-2): 3.762 -> 3.532 | Helped |
| Add classifier guidance to text-infilling MCMC | L-MCMC · One-Billion-Word text infilling | NLL (GPT-2): 4.216 -> 4.146 | Helped |
| Remove the learned prior for actions while retaining the position prior (ablation) | L-MCMC-C · One-Billion-Word constrained generation, k=1 | NLL (GPT-2): 3.762 -> 3.850 | Hurt |
| Remove the learned prior for positions (ablation) | L-MCMC-C · One-Billion-Word constrained generation, k=1 | NLL (GPT-2): 3.762 -> 4.498 | Hurt |
| Train the classifier with one-third of the synthetic dataset (ablation) | L-MCMC-C · One-Billion-Word constrained generation, k=1 | NLL (GPT-2): 3.762 -> 3.921 | Hurt |
| Remove random synthesis from classifier training data (ablation) | L-MCMC-C · One-Billion-Word constrained generation, k=1 | NLL (GPT-2): 3.921 -> 3.954 | Hurt |
| Remove masked-LM synthesis from classifier training data (ablation) | L-MCMC-C · One-Billion-Word constrained generation, k=1 | NLL (GPT-2): 3.921 -> 4.063 | Hurt |
| Increase guided MCMC refinement from 50 to 200 steps | L-MCMC-C · One-Billion-Word constrained generation, k=1 | NLL (GPT-2): 4.425 -> 3.762 | Helped |
| Increase guided XLNet MCMC refinement from 60 to 200 steps | X-MCMC-C · One-Billion-Word constrained generation, k=1 | NLL (GPT-2): 4.152 -> 3.532 | Helped |
| Increase guided MCMC refinement steps | L-MCMC-C and X-MCMC-C · One-Billion-Word constrained generation | Self-BLEU (4-gram): 20.2% -> 33.0%; 12.6% -> 19.0% | Hurt |
| Use the classifier to guide MCMC revision | MCMC-based models · One-Billion-Word constrained generation | acceptance rate | Helped |
Closest papers by meaning
- Constrained Sampling for Language Models Should Be Easy: An MCMC Perspective (NeurIPS 2025 · similarity 0.85)
- Syntactic and Semantic Control of Large Language Models via Sequential Monte Carlo (ICLR 2025 · similarity 0.85)
- Controllable Generation via Locally Constrained Resampling (ICLR 2025 · similarity 0.85)
- COLLIE: Systematic Construction of Constrained Text Generation Tasks (ICLR 2024 · similarity 0.85)
- Principled Gradient-Based MCMC for Conditional Sampling of Text (ICML 2024 · similarity 0.84)
- Guaranteed Generation from Large Language Models (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.