Research map / Large language models / Language, safety and interpretability
Hot or Cold? Adaptive Temperature Sampling for Code Generation with Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AdapT sampling, which uses a higher temperature for challenging code-block-initial tokens and a lower temperature elsewhere, improving pass@k on HumanEval and MBPP for CodeGen-2B, InCoder-6B, and CodeGeeX-13B.
Problem
Standard decoding strategies (greedy, beam, temperature plus top-p sampling) were designed for natural language, and a single temperature is suboptimal for code generation, where some tokens are hard to predict and others are highly deterministic.
Approach
Analysis of per-token loss shows challenging tokens concentrate at the first token of code blocks; AdapT sampling sets a higher temperature a for code-block-initial tokens and a lower temperature b for all other tokens, requiring no training.
Key results
- On HumanEval, AdapT raises CodeGeeX-13B pass@15 from 36.0% to 40.9% (up to 13.6% relative gain over the best temperature baseline).
- AdapT outperforms the best temperature-sampling baseline on pass@5/10/15 for CodeGen-2B, InCoder-6B and CodeGeeX-13B on both HumanEval and MBPP.
- On pass@1, AdapT beats SP-best in 83.3% of cases and is comparable to greedy search.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use adaptive temperature instead of the best fixed-temperature SP setting | CodeGen-2B · HumanEval | pass@5, pass@10, pass@15: pass@5: 33.4 -> 34.4; pass@10: 39.7 -> 40.1; pass@15: 43.9 -> 43.9 | Mixed |
| Use adaptive temperature instead of the best fixed-temperature SP setting | InCoder-6B · HumanEval | pass@5, pass@10, pass@15: pass@5: 25.0 -> 25.8; pass@10: 30.0 -> 31.6; pass@15: 32.9 -> 35.3 | Helped |
| Use adaptive temperature instead of the best fixed-temperature SP setting | CodeGeeX-13B · HumanEval | pass@5, pass@10, pass@15: pass@5: 27.7 -> 29.4; pass@10: 33.0 -> 36.3; pass@15: 36.0 -> 40.9 | Helped |
| Use adaptive temperature instead of the best fixed-temperature SP setting | CodeGen-2B · MBPP | pass@5, pass@10, pass@15: pass@5: 37.1 -> 37.2; pass@10: 43.5 -> 44.4; pass@15: 47.0 -> 48.2 | Helped |
| Use adaptive temperature instead of the best fixed-temperature SP setting | InCoder-6B · MBPP | pass@5, pass@10, pass@15: pass@5: 26.0 -> 26.8; pass@10: 30.6 -> 32.9; pass@15: 34.6 -> 36.8 | Helped |
| Use adaptive temperature instead of the best fixed-temperature SP setting | CodeGeeX-13B · MBPP | pass@5, pass@10, pass@15: pass@5: 25.4 -> 25.8; pass@10: 31.2 -> 32.2; pass@15: 35.6 -> 36.0 | Helped |
| Vary a from 0.2 to 0.9 with b=0.3 (ablation) | AdapT on CodeGeeX-13B · HumanEval | pass@15: 32.9% to 37.8% | Helped |
| Vary b with a=0.8 (ablation) | AdapT on CodeGeeX-13B · HumanEval | pass@1, pass@5, pass@10, pass@15 | Mixed |
Surprising findings
- Increasing a single constant temperature trades off pass@1 against pass@k, and the authors' attempts at linear or exponential decay temperature schedules within statements gave no substantial gain.
Limitations
- Hyperparameters a and b are selected empirically per model and dataset, with no learned schedule.
- Wrong logic remains the most common error type, which decoding alone does not fix.
- Evaluated only on Python benchmarks with three open models in zero-shot setting.
Experimental setup
Datasets: HumanEval MBPP APPS
Benchmarks: HumanEval MBPP
Metrics: pass@1 pass@5 pass@10 pass@15 syntax error counts
Baselines: Greedy search beam search temperature sampling (SP)
Builds on: temperature sampling top-p sampling pass@k
Closest papers by meaning
- Entropy-informed Decoding: Adaptive Information-Driven Branching (ICML 2026 · similarity 0.89)
- Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputs (ICLR 2025 · similarity 0.88)
- Improving Diversity in Language Models: When Temperature Fails, Change the Loss (ICML 2025 · similarity 0.88)
- Improving Open-Ended Text Generation via Adaptive Decoding (ICML 2024 · similarity 0.88)
- Optimizing Temperature for Language Models with Multi-Sample Inference (ICML 2025 · similarity 0.88)
- CodeChemist: Test-Time Scaling for Low-Resource Code Generation via Functional Knowledge Transfer (ICML 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.