Research map / Large language models / Reasoning and chain of thought

CAPER: Clause-Aligned Process Supervision for Text-to-SQL

NeurIPS 2026 · Large language models › Reasoning and chain of thought · approach: Test-time compute scaling · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes CAPER, which derives clause-level process supervision for Text-to-SQL via AST counterfactual intervention and fault injection, trains a 9B clause-level PRM, and uses it as a dense reward in GRPO.

Problem

Text-to-SQL RL uses only terminal execution rewards, which do not localize which intermediate SQL decision caused success or failure; token-level dense rewards are misaligned with SQL semantics and hard to label.

Approach

Automatically derive clause-level preference labels by counterfactually repairing the earliest divergent clause against gold SQL and fault-injecting successful ASTs, assign topology-aware AST node penalties aggregated to clause units, and train a margin-weighted clause PRM used for clause-boundary rewards with clause-level return-to-go GRPO.

Key results

• GRPO-Clause reaches 63.56% EX on BIRD (greedy) and 69.61% under MV@8, versus 59.58% and 60.36% for GPT-5.4 on BIRD.
• CAPER-9B reaches 84.53% top-1 failure-localization accuracy and 90.60% MRR on held-out failures, above Qwen3.5-9B self-debug at 82.78% and 86.35%.
• CAPER-9B as a candidate verifier improves over Majority Vote@8 by +0.7 to +1.6 EX points across four closed-source generators and three splits.

Surprising findings

• Token-level dense reward (GRPO-Token) fails to improve over sparse-reward GRPO and underperforms clause-level reward on most splits.
• Prompted Qwen3.5-9B self-debugging achieves strong localization (82.78% top-1), and the execution-only reward model performs close to random-clause heuristics.

Limitations

• Clause decomposition only at top-level granularity; decisive errors inside a clause (e.g., one predicate or subquery) are not localized.
• PRM trained on specific datasets and may not generalize to other domains or SQL dialects without fine-tuning.
• Annotation pipeline adds computational and engineering overhead over sparse execution rewards.

Experimental setup

Datasets: BIRD-train Spider-dev SynSQL-5k-train

Benchmarks: BIRD dev Spider dev Spider test

Metrics: execution accuracy Majority Vote@8 top-1 failure localization accuracy Hit@3 MRR

Baselines: GPT-5.4 Gemini-2.5-Pro Claude Sonnet 4.6 DeepEye-SQL OmniSQL-7B SQL-R1-7B XiyanSQL-7B AlphaSQL Qwen3.5-9B GRPO-Sparse GRPO-Token Exec-RM-9B

Builds on: GRPO process reward model OmniSQL SQL-R1

Closest papers by meaning

• Beyond Static Pipelines: Learning Dynamic Workflows for Text-to-SQL (ICML 2026 · similarity 0.87)
• SQLens: An End-to-End Framework for Error Detection and Correction in Text-to-SQL (NeurIPS 2025 · similarity 0.86)
• MARS-SQL: A Multi-Agent Reinforcement Learning Framework For Text-To-SQL (ICML 2026 · similarity 0.86)
• Alpha-SQL: Zero-Shot Text-to-SQL using Monte Carlo Tree Search (ICML 2025 · similarity 0.86)
• SQLFixAgent: Towards Semantic-Accurate Text-to-SQL Parsing via Consistency-Enhanced Multi-Agent Collaboration (AAAI 2025 · similarity 0.86)
• CHASE-SQL: Multi-Path Reasoning and Preference Optimized Candidate Selection in Text-to-SQL (ICLR 2025 · similarity 0.85)