Research map / Multi-agent systems and games / LLM agents
RelAgent: LLM Agents as Data Scientists for Relational Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RelAgent, an LLM agent that searches over SQL feature programs and tree-based predictors for relational learning, then deploys the SQL features plus a classical model with no LLM calls at inference.
Problem
Predictive modeling over relational databases requires laborious feature construction and fixed graph/tabular/sequence architectures that may not match task-specific relational structure; the paper targets automating this with interpretable, deployable predictors.
Approach
An LLM agent with exploration, validation, and evaluation-workspace tools iteratively writes SQL feature programs and selects a downstream model during search; the final predictor is the SQL program plus a classical model, executed without LLM calls, so invariance and interpretability come from the query language.
Key results
- On RelBenchV1 entity classification, RelAgent reaches 78.38 average AUROC, second only to KumoRFM-v2 (79.60), with the best average rank (3.17).
- On RelBenchV2 classification, RelAgent averages 87.32 AUROC vs 85.91 for KumoRFM-v2; on 4DBInfer it averages 81.38 vs 79.96.
- Removing the iterative validation feedback loop drops average AUROC from 78.38 to 66.86; using Qwen3-32B as backbone gives 70.13.
Surprising findings
- A rule-only variant that combines SQL features without a supervised predictor reaches only 69.92 AUROC, showing a learned downstream predictor is needed.
Limitations
- Search cost and search reliability: the agent may miss deep joins, high-order relational patterns, or long free-text features within the budget.
- Results depend on the LLM backbone; smaller open models perform substantially worse (Qwen3-32B 70.13).
- Synthetic triangle experiments show the agent sometimes fails to discover the distinctness constraint (no AUROC 1.0 in some runs).
Experimental setup
Datasets: RelBenchV1 RelBenchV2 4DBInfer synthetic directed triangle graphs
Benchmarks: RelBenchV1 RelBenchV2 4DBInfer
Metrics: AUROC normalized MAE average rank
Baselines: LightGBM XGBoost AutoGluon TabPFN-2.5 GraphSAGE HGT RelGNN RelGT Griffin RT_zero KumoRFM-v1 KumoRFM-v2
Builds on: CAAFE LLM-FE ReFuGe Relational Deep Learning CAMEL
Closest papers by meaning
- DS-Agent: Automated Data Science by Empowering Large Language Models with Case-Based Reasoning (ICML 2024 · similarity 0.86)
- RelGNN: Composite Message Passing for Relational Deep Learning (ICML 2025 · similarity 0.85)
- Relational Learning in Pre-Trained Models: A Theory from Hypergraph Recovery Perspective (ICML 2024 · similarity 0.85)
- How to Train Your LLM Web Agent: A Statistical Diagnosis (NeurIPS 2025 · similarity 0.85)
- GraphMaster: Automated Graph Synthesis via LLM Agents in Data-Limited Environments (NeurIPS 2025 · similarity 0.84)
- AutoML-Agent: A Multi-Agent LLM Framework for Full-Pipeline AutoML (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.