Research map / Large language models / Language, safety and interpretability
Improving Diversity in Language Models: When Temperature Fails, Change the Loss
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
The paper analyzes why raising sampling temperature fails to improve recall in LMs, and proposes recall-oriented training losses (TruncR, c-Div, λ-PR) that give a better precision-recall trade-off under temperature scaling.
Problem
Raising decoding temperature is a common way to increase language model diversity, but it often fails to increase recall (coverage of the reference distribution) and can reduce precision, so the precision-recall trade-off cannot be tuned well by temperature alone.
Approach
Theoretical analysis (sparsity bound and an artificial sparse-target case) shows temperature scaling degrades recall at high temperature, so models must be trained for recall; the paper reframes existing losses (Trunc, GOLD, TaiLr) as precision-oriented and proposes recall-oriented variants (TruncR, conditional Tsallis c-Div, and the lambda-PR loss) that reweight NLL.
Key results
- On the integer multiplication task, baseline losses (Trunc, GOLD, TaiLr) favor precision while the proposed recall-oriented losses improve recall at t=1.
- On WritingPrompts with Llama-Alpaca, lambda-PR (gamma=1e-7) raises recall from 4.4 (NLL) to 26.9 at precision 57.1 versus NLL precision 83.3.
- On MathQA-Python with Olmo-1B, the c-Div-trained model reaches precision 0.21 at recall 0.50 vs NLL 0.20 at recall 0.47 at the highest-recall setting.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train with c-Div instead of NLL | Olmo-1B · WritingPrompts | R: 8.9 -> 11.9 | Helped |
| Train with λ-PR (γ=10⁻⁵) instead of NLL | Olmo-1B · WritingPrompts | R: 8.9 -> 12.6 | Helped |
| Train with Trunc-R instead of NLL | Olmo-1B · WritingPrompts | P: 81.4 -> 85.5 | Helped |
| Train with c-Div instead of NLL | Llama-Alpaca · WritingPrompts | R: 4.4 -> 12.6 | Helped |
| Train with λ-PR (γ=10⁻⁷) instead of NLL | Llama-Alpaca · WritingPrompts | R: 4.4 -> 26.9 | Helped |
| Train with c-Div instead of NLL | Llama3.2-3B · WritingPrompts | R: 8.1 -> 17.5 | Helped |
| Train with λ-PR (γ=10⁻⁵) instead of NLL | Llama3.2-3B · WritingPrompts | R: 8.1 -> 19.0 | Helped |
| Train with c-Div instead of NLL | Olmo-1B · MathQA-Python | R: 36.6 -> 46.1 | Helped |
| Train with λ-PR (γ=10⁻⁷) instead of NLL | Olmo-1B · MathQA-Python | R: 36.6 -> 48.1 | Helped |
| Train with c-Div instead of NLL | Llama-Alpaca · MathQA-Python | R: 39.0 -> 43.4 | Helped |
| Train with c-Div instead of NLL | Llama-Alpaca · MathQA-Python | P: 8.8 -> 8.2 | Hurt |
| Decrease decoding temperature below 1 | Language models across evaluated tasks · Various tasks | Precision | Helped |
Surprising findings
- Increasing temperature often reduces recall rather than improving it, contrary to the common intuition that higher temperature boosts diversity.
- The proposed losses can be less stable to train than NLL in practice.
Limitations
- Theory relies on an artificial sparse-target case and assumptions that may not hold for real LMs.
- Trunc and TruncR were omitted for Llama3.1-8B and Llama3.2-3B due to multi-GPU incompatibility.
- The proposed losses can be less stable than NLL during training.
Experimental setup
Datasets: CodeContests WritingPrompts MathQA-Python Alpaca synthetic integer multiplication data
Benchmarks: CodeContests MathQA-Python WritingPrompts integer multiplication
Metrics: precision recall pass@1 pass@100
Baselines: NLL Trunc GOLD TaiLr c-Div
Builds on: Precision-Recall for generative models (Sajjadi et al.) PR-Curve lambda extension (Simon et al.) TaiLr GOLD Trunc
Closest papers by meaning
- Optimizing Temperature for Language Models with Multi-Sample Inference (ICML 2025 · similarity 0.92)
- Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation (NeurIPS 2025 · similarity 0.89)
- A Probabilistic Perspective on Unlearning and Alignment for Large Language Models (ICLR 2025 · similarity 0.89)
- Hot or Cold? Adaptive Temperature Sampling for Code Generation with Large Language Models (AAAI 2024 · similarity 0.88)
- SimpleStrat: Diversifying Language Model Generation with Stratification (NeurIPS 2025 · similarity 0.88)
- Thermometer: Towards Universal Calibration for Large Language Models (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.