Research map / Multi-agent systems and games / LLM agents

Scaling Generalist Data-Analytic Agents

ICLR 2026 · Multi-agent systems and games › LLM agents · approach: Agentic benchmarks and orchestration · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DataMind, a data synthesis and agent training recipe (taxonomy-driven query synthesis, knowledge-augmented trajectory sampling, dynamic SFT+RL objective, sandboxed multi-turn code rollout) that yields DataMind-7B/14B data-analytic agents trained on a 12K trajectory set.

Problem

Open-source LLMs struggle with diverse-format, large-scale data files and long-horizon multi-step data analysis, while existing data-analytic agents rely on prompt engineering over proprietary models and there is little trajectory training data.

Approach

Combine a fine-grained 18-category task taxonomy with recursive easy-to-hard query composition, knowledge-augmented expert trajectory sampling filtered by judge-based self-consistency and rules, and a cosine-annealed mixing coefficient gamma that blends SFT and DAPO RL losses, plus a memory-frugal chunked code rollout.

Key results

• DataMind-14B reaches 71.16% average pass@1 across DABench, TableBench and BIRD, above DeepSeek-V3.1 (70.58%) and GPT-5 (69.44%).
• DataMind-7B reaches 68.10% average pass@1, the best among open-source models in the paper's comparison.
• In the training-strategy ablation on 7B, SFT-and-RL hybrid gives 68.10 pass@1 versus 62.54 for SFT alone, 63.42 for SFT-then-RL and 58.03 for zero-RL.

Surprising findings

• Including all trajectories that reach a consistent answer without best-trajectory selection gave the largest gains, and random selection matched or beat judge-based selection on pass@3.
• Pure zero-RL performed worse than SFT, and SFT-then-RL gave only marginal gains over SFT alone.

Limitations

• Only reasoning-oriented tabular analysis tasks; training, prediction and visualization tasks excluded.
• Backbones restricted to Qwen family (plus one Llama check) with scale capped at 14B.
• RL runs limited to about 350 steps due to data and compute scarcity.

Experimental setup

Datasets: Kaggle CSV/XLSX files BIRD training set OmniSQL synthetic schemas

Benchmarks: DABench TableBench BIRD QRData

Metrics: pass@1 pass@3 Rouge-L

Baselines: GPT-4o o4-mini DeepSeek-R1 DeepSeek-V3.1 GPT-5 QwQ-32B Qwen-2.5-72B TableLLM Table-R1 OmniSQL SQL-R1 Data Interpreter

Builds on: ReAct DAPO GRPO LlamaFactory verl

Closest papers by meaning

• Search More, Think Less: Rethinking Long-Horizon Agentic Search for Efficiency and Generalization (NeurIPS 2026 · similarity 0.87)
• AgentSynth: Scalable Task Generation for Generalist Computer-Use Agents (ICLR 2026 · similarity 0.87)
• Quest: Training Frontier Deep Research Agents with Fully Synthetic Tasks (NeurIPS 2026 · similarity 0.87)
• Star-Agents: Automatic Data Optimization with LLM Agents for Instruction Tuning (NeurIPS 2024 · similarity 0.86)
• FML-bench: A Controlled Study of AI Research Agent Strategies from the Perspective of Search Dynamics (NeurIPS 2026 · similarity 0.86)
• SciAgentGym: Benchmarking Multi-Step Scientific Tool-Use in LLM Agents (ICML 2026 · similarity 0.86)