Research map / Large language models / Language, safety and interpretability

SQLens: An End-to-End Framework for Error Detection and Correction in Text-to-SQL

NeurIPS 2025 · Large language models › Language, safety and interpretability · approach: Controlled text generation · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SQLens, a framework that combines database and LLM error signals, aggregates them via weak supervision to detect clause-level semantic errors in LLM-generated SQL, and corrects queries with an iterative LLM fixer plus an auditor.

Problem

LLM-generated text-to-SQL queries often are syntactically valid but semantically wrong, and existing self-evaluation methods give poor recall and no clause-level localization, while correction methods risk breaking already-correct queries.

Approach

Treat database-based and LLM-based error signals as noisy labeling functions, aggregate them with a generative weak-supervision label model into probabilistic correctness labels to train a classifier, and correct queries step-by-step by fixing the highest-priority error report first, with an LLM auditor that picks between original and revised queries.

Key results

• SQLens outperforms the best LLM self-evaluation baseline by 25.78% in F1 for error detection (DIN-SQL on BIRD: F1 78.88 vs 57.43).
• SQLens improves DIN-SQL execution accuracy by 20.50% on BIRD, while self-reflection gives +15.14%.
• On BIRD with vanilla Claude 3.5 Sonnet prompts, SQLens gains +3.47% accuracy (N_net 50) versus +0.00% for self-reflection.

Measured effects reported in the paper

Surprising findings

• LLM self-reflection breaks the most correct queries, while SQLens corrects more queries with fewer regressions.
• Weak supervision SQLens achieves higher recall and F1 but lower precision and accuracy than the supervised variant.

Limitations

• Inherits limitations of the underlying LLM, such as context window constraints and reasoning limits in applying corrections.
• Iterative aggregation and correction adds latency and overhead, so it is intended as an offline tool.
• Some database signals produce false positives (e.g., suboptimal join tree on correct queries).

Experimental setup

Datasets: BIRD dev Spider dev

Benchmarks: BIRD dev Spider dev

Metrics: execution accuracy gain N_fix N_break N_net accuracy AUC precision recall F1

Baselines: LLM Self-Eval (Bool) LLM Self-Eval (Prob) Self-Reflection Fix-ALL DIN-SQL MAC-SQL CHESS

Builds on: weak supervision label model self-reflection prompting DIN-SQL MAC-SQL CHESS

Closest papers by meaning

• CogSQL: A Cognitive Framework for Enhancing Large Language Models in Text-to-SQL Translation (AAAI 2025 · similarity 0.90)
• SQLFixAgent: Towards Semantic-Accurate Text-to-SQL Parsing via Consistency-Enhanced Multi-Agent Collaboration (AAAI 2025 · similarity 0.89)
• Confidence Estimation for Error Detection in Text-to-SQL Systems (AAAI 2025 · similarity 0.89)
• Structure-Guided Large Language Models for Text-to-SQL Generation (ICML 2025 · similarity 0.89)
• ROUTE: Robust Multitask Tuning and Collaboration for Text-to-SQL (ICLR 2025 · similarity 0.88)
• Alpha-SQL: Zero-Shot Text-to-SQL using Monte Carlo Tree Search (ICML 2025 · similarity 0.88)