Research map / Multi-agent systems and games / LLM agents

CoDA-Bench: Can Code Agents Handle Data-Intensive Tasks?

ICML 2026 · Multi-agent systems and games › LLM agents · approach: Robust and efficient agent execution · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces CoDA-Bench, a 1,009-task benchmark that jointly tests data discovery and code generation for coding agents in large Kaggle-derived Linux sandboxes (about 980 files per environment), and finds the best agent reaches 61.1% execution accuracy.

Problem

Existing benchmarks evaluate code or data-science capabilities in isolation and usually supply all required files explicitly, so they miss whether agents can find relevant data among many distractors in file systems and then write correct code.

Approach

Build data-intensive environments from Kaggle dataset co-occurrence communities (Leiden partitioning) so that distractors are in-distribution, derive verifiable tasks by back-constructing questions from numerical anchors in Kaggle notebook outputs, and refine tasks with an LLM adversarial evolution loop plus human verification.

Key results

• Mini-SWE-agent with GPT-5.5 achieves the highest execution accuracy of 61.1% on CoDA-Bench and 49.6% on CoDA-Hard.
• Even with oracle file paths provided, agents on CoDA-Hard reach only 71.0% average accuracy, while Claude Code (Sonnet-4.6) rises from 45.4% to 73.1%.
• Performance drops as data volume grows, with several communities above 8GB near zero accuracy.

Surprising findings

• Signal-to-noise ratio correlates with accuracy more strongly than raw file count (rho=0.466 vs. -0.271 non-significant).
• Codex CLI with GPT-5.5 uses about 6.8 turns while Mini-SWE-agent with the same model uses 32.5 turns at similar execution accuracy.

Limitations

• Discovery of target data is measured with an exact file-set match, which may be strict.
• Evaluated only on proprietary or API-served models; no open-weight training or fine-tuning experiments.
• Task generation relies on LLM discriminators and generators, which may introduce biases in difficulty calibration.

Experimental setup

Datasets: Kaggle datasets (829 datasets in 31 communities) Kaggle notebooks (30,624 collected, 1,361 selected)

Benchmarks: CoDA-Bench CoDA-Hard

Metrics: Discovery Accuracy Execution Accuracy average turns average tokens cost per task

Baselines: Codex CLI Claude Code Mini-SWE-agent OpenHands

Builds on: Leiden algorithm OpenHands SWE-bench generative adversarial networks

Closest papers by meaning

• GitTaskBench: A Benchmark for Code Agents Solving Real-World Tasks Through Code Repository Leveraging (AAAI 2026 · similarity 0.88)
• EvoCodeBench: Evaluating Coding Agents in Multi-Turn Iterative Interactions (NeurIPS 2026 · similarity 0.87)
• NL2Repo-Bench: Towards Long-Horizon Repository Generation Evaluation of Coding Agents (ICML 2026 · similarity 0.87)
• SlopCodeBench: Benchmarking How Coding Agents Degrade Over Long-Horizon Iterative Tasks (NeurIPS 2026 · similarity 0.87)
• RoadmapBench: Evaluating Long-Horizon Agentic Software Development Across Version Upgrades (NeurIPS 2026 · similarity 0.87)
• DAComp: Benchmarking Data Agents across the Full Data Intelligence Lifecycle (ICLR 2026 · similarity 0.86)