Research map / Multi-agent systems and games / LLM agents

BioXArena: Benchmarking LLM Agents on Multi-Modal Biomedical Machine Learning Tasks

NeurIPS 2026 · Multi-agent systems and games › LLM agents · approach: Agent data synthesis and behavior studies · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces BioXArena, a 76-task, 9-domain biomedical ML coding benchmark where agents must write code to train models and submit held-out predictions, evaluating 11 agent configurations under a shared 2-hour single-GPU sandbox.

Problem

Existing biomedical agent benchmarks either test reasoning/tool use or cover only narrow slices of biomedical ML coding, lacking end-to-end model building on heterogeneous, multi-modal biomedical data with held-out grading.

Approach

A curated benchmark of 76 biomedical ML tasks packaged as public capsules with hidden labels and task-specific biology-aware metrics normalized to 0-1, with agents evaluated end-to-end (code, train, submit) under a common 2-hour single-GPU budget across general LLM coding loops and biomedical/ML research agent scaffolds.

Key results

• MLEvolve with Gemini-3.1-Pro achieves the highest penalized average score of 0.666 across 76 tasks, followed by GPT-5.4 at 0.636.
• Across 836 agent-task runs, 111 fail evaluation, with 52% producing no submission.
• Human experts average 83.9 on a 10-task subset vs. 82.2 for the best agent (GPT-5.4), a 1.7-point gap.

Surprising findings

• Successful submissions are mostly classical models (boosted trees, forests, linear), not neural networks, with neural models concentrated in imaging and sequence/text tasks.
• Specialized biomedical scaffolds do not uniformly dominate general coding LLMs; backbone choice remains a major driver of performance.

Limitations

• Does not cover many biomedical modalities (spatial omics, whole-slide pathology, mass spectrometry, EHR, etc.) and only 76 tasks across 9 domains.
• Excludes very large production-scale tasks requiring multi-day training or distributed compute.
• Single-run evaluation for most agents; only GPT-5.4 and DeepSeek-V3.2 run three times, with high variance for DeepSeek-V3.2.

Experimental setup

Datasets: GTEx v10 ENCODE HuRI CATH OpenProblems PDB PDBbind TCGA METABRIC HAM10000 LIDC AMOS

Benchmarks: BioXArena

Metrics: penalized score (0-1 normalized) Pearson Spearman AUC macro-F1 accuracy AUPRC C-index Dice TM-score

Baselines: Biomni STELLA MLEvolve MLMaster-2.0 GPT-5.4 Claude Opus 4.6 Qwen3.6-Plus Gemini-3.1-Pro GLM-5.1 Gemma-4-31B DeepSeek-V3.2

Builds on: MLE-bench BioML-bench Biomni STELLA MLEvolve

Closest papers by meaning

• BioAgent Bench: An AI Agent Evaluation Suite for Bioinformatics (ICML 2026 · similarity 0.88)
• Omni-Mol: Multitask Molecular Model for Any-to-any Modalities (NeurIPS 2025 · similarity 0.86)
• Mol-Instructions: A Large-Scale Biomolecular Instruction Dataset for Large Language Models (ICLR 2024 · similarity 0.85)
• BioBridge: Bridging Biomedical Foundation Models via Knowledge Graphs (ICLR 2024 · similarity 0.85)
• Bio-xLSTM: Generative modeling, representation and in-context learning of biological and chemical sequences (ICLR 2025 · similarity 0.85)
• PyTDC: A multimodal machine learning training, evaluation, and inference platform for biomedical foundation models (ICML 2025 · similarity 0.84)