Research map / Large language models / Language, safety and interpretability
Who Knows the Answer? Finding the Best Model and Prompt for Each Query Using Confidence-Based Search
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PICS, which predicts the semantic confidence of each LLM-prompt template pair per query with a cheap regression ensemble, then routes the query to the most confident pair and returns the most confident sampled response.
Problem
Choosing which LLM and which prompt template to use for each individual input, without calling every model-prompt pair (which would require many LLM calls) to estimate response confidence.
Approach
Replace costly sampling-based semantic confidence with a regression ensemble that predicts each model-prompt pair's confidence from features built from nearest-neighbor training-query confidences and their embedding distances; select the pair with highest predicted confidence, then sample responses from it and return the most confident one.
Key results
- On Question Formation with MPT-30B, PICS-CS raises accuracy from 42.00% (Most Confident) to 79.50% in the single-model setting (PICS-RS 45.65%); the table lists 83.20% for OCS-CS.
- Across all four models, PICS-CS improves over the average model-prompt pair accuracy on most datasets, e.g. TriviaQA multi-model accuracy 33.58% (MPT Avg.) to 51.35% (PICS-CS).
- In the multi-model setting, PICS-CS is best on StrategyQA (70.55%) and TriviaQA (51.35%), but on Question Formation it trails Most Confident (96.80% vs 99.10%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use instance-level predicted-confidence search with confidence sampling instead of average model-prompt performance | PICS on all models · StrategyQA | accuracy: 41.70 -> 70.55 | Helped |
| Use instance-level predicted-confidence search with confidence sampling instead of average model-prompt performance | PICS on all models · TriviaQA | accuracy: 33.58 -> 51.35 | Helped |
| Use instance-level predicted-confidence search with confidence sampling instead of average model-prompt performance | PICS on all models · Q. Formation | accuracy: 65.85 -> 96.80 | Helped |
| Add confidence sampling rather than returning a single randomly sampled response (ablation) | PICS with FLAN-T5-XL · StrategyQA | accuracy: 49.20 -> 53.10 | Helped |
| Add confidence sampling rather than returning a single randomly sampled response (ablation) | PICS with FLAN-T5-XL · TriviaQA | accuracy: 21.40 -> 27.10 | Helped |
| Add confidence sampling rather than returning a single randomly sampled response (ablation) | PICS with FLAN-T5-XL · Q. Formation | accuracy: 66.25 -> 73.45 | Helped |
| Add confidence sampling rather than returning a single randomly sampled response (ablation) | PICS across all models · StrategyQA | accuracy: 69.00 -> 70.55 | Helped |
| Add confidence sampling rather than returning a single randomly sampled response (ablation) | PICS across all models · TriviaQA | accuracy: 48.35 -> 51.35 | Helped |
| Add confidence sampling rather than returning a single randomly sampled response (ablation) | PICS across all models · Q. Formation | accuracy: 97.50 -> 96.80 | Hurt |
| Search instances across models and prompts using PICS-CS instead of using the most confident fixed pair | PICS · Q. Formation | accuracy: 99.10 -> 96.80 | Hurt |
Surprising findings
- Predicted-confidence search sometimes outperforms search using the directly computed semantic confidence, which the authors attribute to regularization/smoothing of outliers.
- Model size does not consistently predict accuracy; smaller instruction-tuned models sometimes beat larger ones on StrategyQA.
Limitations
- Question Formation performance falls below Most Confident in the multi-model setting when poorly performing pairs are erroneously selected.
- Evaluated on only three datasets and four models with six 3-shot prompts per dataset.
- Final confidence sampling requires multiple LLM calls for the selected pair, adding inference cost.
Experimental setup
Datasets: StrategyQA TriviaQA Question Formation (BIG-bench)
Benchmarks: StrategyQA TriviaQA Question Formation
Metrics: thresholded RougeL accuracy runtime
Baselines: MPT Avg. Most Confident OCS-RS OCS-CS
Builds on: semantic entropy / semantic confidence (Kuhn et al. 2023) Lin, Trivedi and Sun 2023 similarity-graph confidence Sentence Transformers
Closest papers by meaning
- Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting (ICLR 2024 · similarity 0.89)
- On the Worst Prompt Performance of Large Language Models (NeurIPS 2024 · similarity 0.89)
- Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs (ICLR 2024 · similarity 0.88)
- Learning to Route LLMs with Confidence Tokens (ICML 2025 · similarity 0.88)
- Large Language Models Must Be Taught to Know What They Don’t Know (NeurIPS 2024 · similarity 0.88)
- Prompt Sketching for Large Language Models (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.