Research map / Large language models / Agents, code and math
Alpha-SQL: Zero-Shot Text-to-SQL using Monte Carlo Tree Search
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Alpha-SQL performs zero-shot Text-to-SQL by running Monte Carlo Tree Search over SQL construction actions, with an LLM as the action model and a self-consistency execution-based reward, without fine-tuning.
Problem
Zero-shot Text-to-SQL with LLMs struggles to map natural language questions and diverse database schemas to correct SQL without task-specific fine-tuning, especially for complex queries.
Approach
Model SQL generation as a tree search where nodes are partial reasoning states and edges are SQL construction actions chosen and executed by an LLM (LLM-as-Action-Model) inside MCTS, scored by a self-supervised execution self-consistency reward instead of a trained reward model.
Key results
- Alpha-SQL with Qwen2.5-Coder-32B reaches 69.7% execution accuracy on BIRD dev, beating the best prior zero-shot GPT-4o method RSL-SQL (67.2%) by 2.5 points.
- Alpha-SQL with Qwen2.5-Coder-7B reaches 66.8% on BIRD dev, and 84.0% (7B) / 87.0% (14B) on Spider dev.
- On the SDS subset, Alpha-SQL raises Qwen2.5-Coder-7B from 47.6% to 64.6% and Phi-4 from 43.5% to 60.0%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply Alpha-SQL to Qwen2.5-Coder-7B | Qwen2.5-Coder-7B · BIRD SDS | accuracy: 47.6 -> 64.6 | Helped |
| Apply Alpha-SQL to Phi-4 | Phi-4 · BIRD SDS | accuracy: 43.5 -> 60.0 | Helped |
| Increase inference model size from Qwen2.5-Coder-7B to Qwen2.5-Coder-32B | Alpha-SQL · BIRD development | execution accuracy (All, %): 66.8 -> 69.7 | Helped |
| Increase inference model size from Qwen2.5-Coder-7B to Qwen2.5-Coder-14B | Alpha-SQL · Spider development | execution accuracy (All, %): 84.0 -> 87.0 | Helped |
| Add question rephrasing (ablation) | Alpha-SQL · BIRD SDS | accuracy: 63.9 -> 64.6 | Helped |
| Add schema selection (ablation) | Alpha-SQL · BIRD SDS | accuracy: 63.1 -> 64.6 | Helped |
| Add column value identification (ablation) | Alpha-SQL · BIRD SDS | accuracy: 64.2 -> 64.6 | Helped |
| Add column function identification (ablation) | Alpha-SQL · BIRD SDS | accuracy: 64.0 -> 64.6 | Helped |
| Add SQL revision (ablation) | Alpha-SQL · BIRD SDS | accuracy: 62.8 -> 64.6 | Helped |
| Increase the number of MCTS rollouts | Alpha-SQL with Qwen2.5-Coder-7B · BIRD SDS | final accuracy | Helped |
Surprising findings
- Plain Gemini-2.0-Flash-Thinking-Exp and DeepSeek-R1 without the framework score lower than Alpha-SQL built on a 7B coder model on the SDS subset.
Limitations
- Relies on repeated LLM sampling and SQL execution, which increases inference cost.
- Self-consistency reward can favor consistent but wrong answers when samples agree on a flawed query.
- Some comparisons use results reported from other papers and an SDS subset of only 147 samples.
Experimental setup
Datasets: BIRD dev Spider dev
Benchmarks: BIRD dev Spider dev BIRD Subsampled Development Set (SDS)
Metrics: execution accuracy
Baselines: CHASE-SQL RSL-SQL DAIL-SQL SuperSQL MCS-SQL XiYan-SQL CodeS C3-SQL DIN-SQL MAC-SQL GPT-4o DeepSeek-R1
Builds on: Monte Carlo Tree Search rStar CHASE-SQL CHESS-SQL UCT
Closest papers by meaning
- ROUTE: Robust Multitask Tuning and Collaboration for Text-to-SQL (ICLR 2025 · similarity 0.89)
- CHASE-SQL: Multi-Path Reasoning and Preference Optimized Candidate Selection in Text-to-SQL (ICLR 2025 · similarity 0.89)
- Structure-Guided Large Language Models for Text-to-SQL Generation (ICML 2025 · similarity 0.89)
- LogicCat: A Chain-of-Thought Text-to-SQL Benchmark for Complex Reasoning (AAAI 2026 · similarity 0.88)
- CogSQL: A Cognitive Framework for Enhancing Large Language Models in Text-to-SQL Translation (AAAI 2025 · similarity 0.88)
- MARS-SQL: A Multi-Agent Reinforcement Learning Framework For Text-To-SQL (ICML 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.