Research map / Large language models / Language, safety and interpretability
Calibrate Before Use: Improving Few-shot Performance of Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply contextual calibration to GPT-3 2.7B | GPT-3 2.7B few-shot prompting · SST-2, 4-shot | accuracy: 59.1_{ 10.2} -> 79.9_{ 7.8} | Helped |
| Apply contextual calibration to GPT-3 2.7B | GPT-3 2.7B few-shot prompting · DBPedia, 1-shot | accuracy: 25.9_{ 4.4} -> 61.6_{ 2.9} | Helped |
| Apply contextual calibration to GPT-3 2.7B | GPT-3 2.7B few-shot prompting · AGNews, 4-shot | accuracy: 43.3_{ 8.3} -> 71.1_{ 8.5} | Helped |
| Apply contextual calibration to GPT-3 2.7B | GPT-3 2.7B few-shot prompting · LAMA, 4-shot | accuracy: 35.8_{ 3.8} -> 37.4_{ 3.4} | Helped |
| Apply contextual calibration to GPT-3 175B | GPT-3 175B few-shot prompting · RTE, 1-shot | accuracy: 62.9_{ 2.7} -> 62.8_{ 2.3} | No clear effect |
| Apply contextual calibration to GPT-3 175B | GPT-3 175B few-shot prompting · SST-2, 8-shot | accuracy: 95.6_{ 1.0} -> 95.3_{ 0.7} | No clear effect |
| Apply contextual calibration to GPT-2 | GPT-2 few-shot prompting · CB, 0-shot | accuracy: 44.6_{ \text{0.0}} -> 17.9_{ \text{0.0}} | Hurt |
| Apply contextual calibration to GPT-2 | GPT-2 few-shot prompting · DBPedia, 0-shot | accuracy: 64.3_{ \text{0.0}} -> 58.3_{ \text{0.0}} | Hurt |
| Use N/A alone instead of the three-input ensemble (ablation) | Contextual calibration on SST-2 · SST-2, 1-shot | accuracy: 79.0 -> 74.2 | Hurt |
| Use [MASK] alone instead of the three-input ensemble (ablation) | Contextual calibration on SST-2 · SST-2, 1-shot | accuracy: 79.0 -> 74.5 | Hurt |
| Use the empty string alone instead of the three-input ensemble (ablation) | Contextual calibration on SST-2 · SST-2, 1-shot | accuracy: 79.0 -> 72.9 | Hurt |
| Use the man. instead of the three-input ensemble (ablation) | Contextual calibration on SST-2 · SST-2, 1-shot | accuracy: 79.0 -> 79.4 | No clear effect |
Closest papers by meaning
- Batch Calibration: Rethinking Calibration for In-Context Learning and Prompt Engineering (ICLR 2024 · similarity 0.90)
- The Surprising Effectiveness of Test-Time Training for Few-Shot Learning (ICML 2025 · similarity 0.89)
- C-TPT: Calibrated Test-Time Prompt Tuning for Vision-Language Models via Text Feature Dispersion (ICLR 2024 · similarity 0.87)
- LitCab: Lightweight Language Model Calibration over Short- and Long-form Responses (ICLR 2024 · similarity 0.87)
- Calibrating Large Language Models with Sample Consistency (AAAI 2025 · similarity 0.87)
- Your Pre-trained LLM is Secretly an Unsupervised Confidence Calibrator (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.