Research map / Large language models / Reasoning and chain of thought
Optimizing Temperature for Language Models with Multi-Sample Inference
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TURN, an entropy-based method that predicts the near-optimal sampling temperature for multi-sample aggregation (majority voting, best-of-N) without labeled validation data, by locating an entropy turning point.
Problem
Choosing the sampling temperature for multi-sample aggregation (majority voting, best-of-N) in LLMs usually requires labeled validation data or uses a fixed default temperature, which is suboptimal across models, tasks, and aggregation methods.
Approach
Token-level entropy of model generations versus temperature shows a sharp spike; the entropy turning point (where log-entropy changes from concave to convex) marks where sample quality collapses, and setting temperature there (plus an aggregation-specific offset) gives near-optimal accuracy without labels. A stochastic process model is used to explain the spike.
Key results
- TURN achieves hit rates of 12/13 on MATH (majority voting) and 11/13 on MBPP (best-of-N), with average performance drops of 0.32% and 0.59% relative to grid-searched best temperature.
- Averaged over both tasks, TURN's performance drop is 0.4% versus 1.15% for the best fixed temperature (0.7), and the correlation between EntP-predicted accuracy and best grid accuracy is 0.9998 on MATH.
- Optimal temperature correlates negatively with model-task distance (entropy-based), with coefficients -0.895 on MATH and -0.777 on MBPP.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use TURN instead of the strongest fixed temperature baseline | Multi-sample temperature selection · MATH, majority voting | Avg. Drop: 0.8% -> 0.3% | Helped |
| Use TURN instead of the strongest fixed temperature baseline | Multi-sample temperature selection · MBPP, best-of-N | Avg. Drop: 0.9% -> 0.5% | Helped |
| Add aggregation adaptation factor β (ablation) | TURN · MBPP, best-of-N | Avg. Drop: 1.1% -> 0.5% | Helped |
| Add aggregation adaptation factor β (ablation) | TURN · MATH, majority voting | Avg. Drop: 0.3% -> 0.3% | No clear effect |
| Increase temperature-estimation sample size from 10 to 40 (ablation) | TURN on Llama-3.1-8B-Instruct · MATH | Performance Drop: 0.9% -> 0.2% | Helped |
| Increase temperature-estimation sample size from 40 to 100 (ablation) | TURN on Llama-3.1-8B-Instruct · MATH | Performance Drop: 0.2% -> 0.1% | Helped |
| Increase temperature-estimation sample size from 100 to 800 (ablation) | TURN on Llama-3.1-8B-Instruct · MATH | Performance Drop: 0.1% -> 0.0% | Helped |
Surprising findings
- Optimal temperature for task-finetuned models is higher (around 0.9-1.1) while general-purpose models favor lower temperatures (0.5-0.7).
Limitations
- Aggregation adaptation factor (+0.1 for best-of-N) is calibrated on MATH and transferred to MBPP.
- Evaluation uses small subsets (200 MATH problems, 100 MBPP problems) and 13 models.
- Stochastic process model is a simplified toy model with assumed token categories.
Experimental setup
Datasets: MATH MBPP
Benchmarks: MATH MBPP
Metrics: accuracy (majority voting) pass@K hit rate temperature gap performance drop
Baselines: fixed temperature (0.1, 0.3, 0.5, 0.7, 0.9, 1.1)
Builds on: self-consistency best-of-N sampling Easy-to-Hard generalization vLLM
Closest papers by meaning
- Improving Diversity in Language Models: When Temperature Fails, Change the Loss (ICML 2025 · similarity 0.92)
- Thermometer: Towards Universal Calibration for Large Language Models (ICML 2024 · similarity 0.89)
- A Probabilistic Perspective on Unlearning and Alignment for Large Language Models (ICLR 2025 · similarity 0.89)
- Hot or Cold? Adaptive Temperature Sampling for Code Generation with Large Language Models (AAAI 2024 · similarity 0.88)
- Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation (NeurIPS 2025 · similarity 0.87)
- Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputs (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.