Research map / Reinforcement learning / Visual and representation RL
BIRD-RL: Scaling Agentic Reinforcement Learning over Stateful Data-Centric Environments
Ge Qu, Jinyang Li, Xiaohan Xu, Xiaolong Li, Bowen Qin, Nan Huo, Ziwei Tang, Chenhao Ma, Reynold Cheng
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BIRD-RL, an agentic reinforcement learning approach for language agents operating over live, stateful database environments, where success requires reasoning about evolving database states and repairing state-mutating operations.
Problem
Language agents must operate over live, stateful database environments, where success depends on reasoning about evolving database state, using execution feedback, and repairing state-mutating operations rather than only generating initial SQL.
Approach
Scales agentic reinforcement learning over stateful data-centric environments; the abstract is truncated, so the specific mechanism is not stated.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Stage 4 stateful interactive agentic RL after Stages 1-3 (ablation) | BIRD-Talon-7B · BIRD-CRITIC-SQLite | Success Rate (%): 40.60 -> 44.40 | Improved |
| add Stage 3 adaptive reasoning RL on top of Stages 1-2 (ablation) | BIRD-Talon-7B · BIRD-CRITIC-SQLite | Success Rate (%): 39.20 -> 40.60 | Improved |
| add Stage 2 tool-use imitation on top of Stage 1 (ablation) | BIRD-Talon-7B · BIRD-CRITIC-SQLite | Success Rate (%): 35.20 -> 39.20 | Improved |
| use purely execution-based reward instead of adding partial-correctness proxy reward (ablation) | BIRD-Talon-7B Stage 4 · BIRD-CRITIC-SQLite | Success Rate (%): 42.80 -> 44.40 | Improved |
| use purely execution-based reward instead of LLM-as-a-judge proxy reward (ablation) | BIRD-Talon-7B Stage 4 · BIRD-CRITIC-SQLite | Success Rate (%): 41.40 -> 44.40 | Improved |
| use purely execution-based reward instead of heuristic rubric proxy reward (ablation) | BIRD-Talon-7B Stage 4 · BIRD-CRITIC-SQLite | Success Rate (%): 42.00 -> 44.40 | Improved |
| replace per-rollout containers with trajectory-scoped reusable database pool | BIRD-RL infrastructure · SQLite rollout, B=16, R=16 | Online database I/O: 2.94 TiB -> 236 GiB (12.8x) | Improved |
| apply BIRD-RL training on top of SQL-Act scaffold and supervised BIRD-Fixer | Qwen2.5-Coder-7B · BIRD-CRITIC-SQLite | Success Rate (%): 33.60 -> 44.40 | Improved |
| use SQL-Act multi-turn scaffold alone with untrained base model | Qwen2.5-Coder-7B · BIRD-CRITIC-SQLite | Success Rate (%): 26.20 -> 29.80 | Improved |
| use multi-turn agentic inference with proprietary models instead of single-turn | General-purpose LLMs · BIRD-CRITIC-SQLite | Success Rate (%): Claude-Opus-4.6 46.20 -> 48.20; Qwen3-Coder-480B 41.60 -> 42.60 | Mixed |
| use debug-heavy 3:1 debugging-to-generation data mix rather than balanced or generation-heavy (ablation) | BIRD-Zeno-7B · BIRD-CRITIC-SQLite and BIRD-Mini | Success Rate / Execution Accuracy | Mixed |
Closest papers by meaning
- MARS-SQL: A Multi-Agent Reinforcement Learning Framework For Text-To-SQL (ICML 2026 · similarity 0.90)
- AstraFlow: Dataflow-Oriented Reinforcement Learning for Agentic LLMs (NeurIPS 2026 · similarity 0.89)
- Generalizable End-to-End Tool-Use RL with Synthetic CodeGym (ICLR 2026 · similarity 0.87)
- AREAL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoning (NeurIPS 2025 · similarity 0.87)
- SQL-R1: Training Natural Language to SQL Reasoning Model By Reinforcement Learning (NeurIPS 2025 · similarity 0.87)
- Learning Agentic Policy from Action Guidance (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
