Research map / Large language models / Language, safety and interpretability
A Contrastive Framework for Neural Text Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add contrastive loss to MLE training | GPT-2-small · Wikitext-103 test | ppl: 24.32 -> 23.82 | Helped |
| Add contrastive loss to MLE training | GPT-2-small · Wikitext-103 test | accuracy: 39.63 -> 40.91 | Helped |
| Use unlikelihood training instead of MLE | GPT-2-small · Wikitext-103 test | ppl: 24.32 -> 28.57 | Hurt |
| Use contrastive search instead of nucleus sampling | SimCTG GPT-2-small · Wikitext-103 test | coherence: 0.584 -> 0.610 | Helped |
| Use contrastive search instead of nucleus sampling | SimCTG GPT-2-small · Wikitext-103 test | gen-ppl: 47.19 -> 18.26 | Helped |
| Add contrastive training before contrastive decoding | GPT-2-small on Wikitext-103 · Wikitext-103 human evaluation | coherence: 2.78 -> 3.25 | Helped |
| Use contrastive search instead of nucleus sampling | SimCTG · LCCC | fluency: 3.58 -> 3.96 | Helped |
| Use contrastive search instead of nucleus sampling | SimCTG · DailyDialog | coherence: 2.75 -> 3.73 | Helped |
| Increase GPT-2 model size to GPT-2-large | SimCTG + contrastive search · Wikitext-103 human evaluation | coherence: 3.25 -> 3.33 | Helped |
| Set contrastive loss margin to 0.5 rather than too small or too large (ablation) | SimCTG training · Wikitext-103 test | ppl | Mixed |
| Increase alpha in contrastive search (ablation) | SimCTG decoding · Wikitext-103 test | diversity | Helped |
| Increase k in contrastive search (ablation) | SimCTG decoding · Wikitext-103 test | diversity | Helped |
Closest papers by meaning
- Reflection-Window Decoding: Text Generation with Selective Refinement (ICML 2025 · similarity 0.89)
- Closing the Curious Case of Neural Text Degeneration (ICLR 2024 · similarity 0.89)
- Improving Open-Ended Text Generation via Adaptive Decoding (ICML 2024 · similarity 0.88)
- Decoding Game: On Minimax Optimality of Heuristic Text Generation Strategies (ICLR 2025 · similarity 0.88)
- Compressed and Smooth Latent Space for Text Diffusion Modeling (NeurIPS 2025 · similarity 0.87)
- TEncDM: Understanding the Properties of the Diffusion Model in the Space of Language Model Encodings (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.