Research map / Large language models / Language, safety and interpretability

CogSQL: A Cognitive Framework for Enhancing Large Language Models in Text-to-SQL Translation

AAAI 2025 · Large language models › Language, safety and interpretability · approach: LLM unlearning and benchmarks · 7 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes COGSQL, a training-light text-to-SQL framework that mimics human cognition via schema/keyword recall, concept-enhanced chain-of-thought prompting, and NLQ/result consistency correction on top of LLM prompting.

Problem

LLM-based text-to-SQL methods typically target isolated aspects (NLQ understanding or SQL quality), missing a holistic reasoning process; the paper asks whether mimicking human cognitive steps improves accuracy on cross-domain text-to-SQL benchmarks.

Approach

A three-module pipeline: coarse-to-fine schema linking (cross-encoder plus LLM table selection) and syntax keyword prediction (classifiers trained with LLM-based sample-centric augmentation), followed by two-stage concept-enhanced CoT (NLQ interpretation then SQL composition) and consistency-based correction (NLQ consistency via reverse question generation and result consistency via grammar rules

Key results

• COGSQL + GPT4 reaches 59.58% EX on BIRD dev and 64.30 VES, versus 58.93 EX / 63.07 VES for the best baseline DEA-SQL + GPT4.
• COGSQL with GPT-4o-mini lifts BIRD dev EX from 49.61% to 56.26%, and DeepSeek-Coder-V2 from 53.46% to 56.52%.
• On Spider-Realistic, GPT-4 + COGSQL achieves 85% EX versus 73.4% for the GPT4 baseline; the framework trails SUPER-SQL and DEA-SQL on Spider test.

Measured effects reported in the paper

Surprising findings

• Removing the concept-enhanced CoT module causes the largest BIRD drop (56.26 to 52.41), while removing coarse-to-fine schema linking causes the largest Spider drop among ablations.
• Keyword addition augmentation helps classifiers more than NLQ rewriting, per AUC ablation.

Limitations

• Relies on proprietary GPT models for the main results, making reproducibility and cost dependent on API access.
• Fixed two-shot demonstrations rather than retrieval-based selection; NLQ-specific demonstration selection left to future work.
• Schema misuse remains the most common error after COGSQL (75 cases on 500 BIRD NLQs).

Experimental setup

Datasets: Spider Spider-DK Spider-Realistic Spider-Syn BIRD

Benchmarks: Spider Spider-DK Spider-Realistic Spider-Syn BIRD

Metrics: Execution Accuracy Valid Efficiency Score

Baselines: RESDSQL CODES DIN-SQL MAC-SQL TA-SQL DAIL-SQL DEA-SQL SUPER-SQL GPT4

Builds on: RESDSQL CODES DIN-SQL MAC-SQL DAIL-SQL

Closest papers by meaning

• Structure-Guided Large Language Models for Text-to-SQL Generation (ICML 2025 · similarity 0.90)
• ROUTE: Robust Multitask Tuning and Collaboration for Text-to-SQL (ICLR 2025 · similarity 0.90)
• SQLens: An End-to-End Framework for Error Detection and Correction in Text-to-SQL (NeurIPS 2025 · similarity 0.90)
• SQL-R1: Training Natural Language to SQL Reasoning Model By Reinforcement Learning (NeurIPS 2025 · similarity 0.89)
• Alpha-SQL: Zero-Shot Text-to-SQL using Monte Carlo Tree Search (ICML 2025 · similarity 0.88)
• Chain-of-Knowledge: Grounding Large Language Models via Dynamic Knowledge Adapting over Heterogeneous Sources (ICLR 2024 · similarity 0.88)