Research map / Large language models / Language, safety and interpretability
The Curious Case of Neural Text Degeneration
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use nucleus sampling with p=0.95 instead of top-k sampling with k=640 | GPT-2 Large · WebText held-out continuations | HUSE: 0.94 -> 0.97 | Helped |
| Use nucleus sampling with p=0.95 instead of pure sampling | GPT-2 Large · WebText held-out continuations | Perplexity: 22.73 -> 13.13 | Helped |
| Use nucleus sampling with p=0.95 instead of beam search | GPT-2 Large · WebText held-out continuations | HUSE | Helped |
| Use beam search or greedy decoding rather than stochastic sampling | GPT-2 Large · Open-ended text generation | Repetition: 73.66; 28.94 | Hurt |
| Sample directly from the full predicted distribution without truncation | GPT-2 Large · WebText held-out continuations | Perplexity: 22.73 | Hurt |
| Use low-temperature top-k sampling (k=40, t=0.7) | GPT-2 Large · WebText held-out continuations | Repetition: 8.86 | Hurt |
| Use top-k sampling with k=640 rather than nucleus sampling | GPT-2 Large · Open-ended text generation | Perplexity: 13.13 -> 13.82 | Mixed |
Closest papers by meaning
- Closing the Curious Case of Neural Text Degeneration (ICLR 2024 · similarity 0.90)
- Decoding Game: On Minimax Optimality of Heuristic Text Generation Strategies (ICLR 2025 · similarity 0.90)
- Language Model Decoding as Direct Metrics Optimization (ICLR 2024 · similarity 0.90)
- Reflection-Window Decoding: Text Generation with Selective Refinement (ICML 2025 · similarity 0.89)
- Top-H Decoding: Adapting the Creativity and Coherence with Bounded Entropy in Text Generation (NeurIPS 2025 · similarity 0.89)
- Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputs (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.