Research map / Large language models / Agents, code and math
Beyond Text-to-SQL: Can LLMs Really Debug Enterprise ETL SQL?
Jing Ye, Yiwen Duan, Yonghong Yu, Xinpei Zhao, Yang Gao, Xing Chen
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| fine-tune Qwen-2.5-Coder-7B on in-domain SQL debugging data (vanilla SFT) | Qwen-2.5-Coder-7B · Squirrel-Syntax | GM score: 8.96 -> 30.70 | Improved |
| fine-tune the 7B model with in-domain SFT instead of using the 32B model | Qwen-2.5-Coder · Squirrel-Syntax | GM score: 20.26 (32B) -> 30.70 (7B + SFT) | Improved |
| fine-tune Qwen-2.5-Coder-7B with vanilla SFT | Qwen-2.5-Coder-7B · Squirrel-Semantic | GM score: 7.75 -> 15.70 | Improved |
| mask loss on unchanged tokens (DM-SFT) instead of full-loss SFT (ablation) | Qwen-2.5-Coder-7B SFT · Squirrel-Syntax | GM score: 30.70 -> 33.18 | Improved |
| predict only diff segments (Diff-SFT) instead of full SQL (ablation) | Qwen-2.5-Coder-7B SFT · Squirrel-Syntax | GM score: 30.70 -> 22.81 | Mixed |
| use reasoning models rather than non-reasoning models | LLMs · Squirrel-Syntax and Squirrel-Semantic | GM score / MB | Improved |
| use Text-to-SQL specialized model (OmniSQL-32B) for debugging | OmniSQL-32B · Squirrel-Syntax | GM score: 6.40 GM despite 87.6% on Spider | Hurt |
| Kimi-K2 main agent plus Qwen3-Coder sub-agent with TQS feedback | Kimi-K2 · Squirrel-Syntax | EM / GM score: EM 14.07 -> 23.24; GM 27.72 -> 28.78 | Improved |
| GPT-4o main agent plus Qwen3-Coder sub-agent | GPT-4o · Squirrel-Syntax | GM score: GM 4.69 -> 14.37 | Mixed |
| use multi-round agent modification | agent systems · Squirrel-Syntax | MB score: Qwen3-Coder-480B 61.62 -> 49.89 (Kimi-K2 + Qwen3-Coder agent) | Hurt |
| give explicit error messages (Syntax vs Semantic task) | LLMs · Squirrel | GM score: Claude-4-Sonnet 32.17 -> 36.46 | Improved |
| train for more steps beyond about 400 on vanilla SFT (ablation) | Qwen-2.5-Coder-7B SFT · Squirrel-Syntax | performance | No clear effect |
Closest papers by meaning
- SWE-SQL: Illuminating LLM Pathways to Solve User SQL Issues in Real-World Applications (NeurIPS 2025 · similarity 0.91)
- Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows (ICLR 2025 · similarity 0.88)
- PARROT: A Benchmark for Evaluating LLMs in Cross-System SQL Translation (NeurIPS 2025 · similarity 0.88)
- SQLens: An End-to-End Framework for Error Detection and Correction in Text-to-SQL (NeurIPS 2025 · similarity 0.86)
- Structure-Guided Large Language Models for Text-to-SQL Generation (ICML 2025 · similarity 0.85)
- CoRe: Benchmarking LLMs’ Code Reasoning Capabilities through Static Analysis Tasks (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
