Research map / Large language models / Language, safety and interpretability
Confidence Estimation for Text-to-SQL in Large Language Models
Sepideh Entezari Maleki, Mohammadreza Pourreza, Davood Rafiei
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use SQL-aware token filtering and folding instead of full-token confidence (average aggregation) | logit-based confidence, GPT-3.5 · BIRD dev | AUC-ROC: 74.88 -> 79.06 | Improved |
| use SAC product aggregation instead of FTC product | logit-based confidence, GPT-3.5 · Spider dev | AUC-ROC: 68.10 -> 71.66 | Improved |
| use SAC average on Spider instead of product aggregation | SAC, GPT-3.5 · Spider dev | AUC-ROC: 71.66 -> 65.01 | Hurt |
| use product aggregation instead of average for SAC on BIRD | SAC, GPT-3.5 · BIRD dev | ECE: 12.15 -> 22.88 | Hurt |
| use execution-based consistency instead of embedding-based consistency | black-box confidence, GPT-3.5 · BIRD dev | AUC-ROC: 61.21 -> 67.36 | Improved |
| use white-box SAC instead of best black-box (execution consistency) | confidence estimation, GPT-3.5 · BIRD dev | AUC-ROC: 67.36 -> 79.06 | Improved |
| use verbalized confidence (vanilla) on BIRD | GPT-3.5 · BIRD dev | ECE: 58.61 | Hurt |
| add chain-of-thought to verbalized confidence | GPT-3.5 · BIRD dev | AUC-ROC: 55.67 -> 57.12 | Improved |
| add execution results to CoT verbalized confidence | GPT-3.5 · BIRD dev | AUC-ROC: 57.12 -> 57.73 | Improved |
| add token exclusion to SAC-Average (ablation) | SAC-Average · BIRD dev | AUC-ROC: -5.7% when removed | Improved |
| add case folding to SAC-Average (ablation) | SAC-Average · BIRD dev | AUC-ROC: -3.8% when removed | Improved |
| add synonym folding to SAC-Average (ablation) | SAC-Average · BIRD dev | AUC-ROC: -0.5% when removed | Improved |
Closest papers by meaning
- Confidence Estimation for Error Detection in Text-to-SQL Systems (AAAI 2025 · similarity 0.92)
- SQLens: An End-to-End Framework for Error Detection and Correction in Text-to-SQL (NeurIPS 2025 · similarity 0.89)
- ROUTE: Robust Multitask Tuning and Collaboration for Text-to-SQL (ICLR 2025 · similarity 0.89)
- CogSQL: A Cognitive Framework for Enhancing Large Language Models in Text-to-SQL Translation (AAAI 2025 · similarity 0.88)
- Alpha-SQL: Zero-Shot Text-to-SQL using Monte Carlo Tree Search (ICML 2025 · similarity 0.88)
- Structure-Guided Large Language Models for Text-to-SQL Generation (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
