Research map / Multi-agent systems and games / LLM agents

CoMind: Towards Community-Driven Agents for Machine Learning Engineering

ICLR 2026 · Multi-agent systems and games › LLM agents · approach: Agent data synthesis and behavior studies · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces MLE-Live, a live evaluation framework simulating a Kaggle community with time-stamped discussions and kernels, and CoMind, a five-role multi-agent system with parallel coding agents that reaches 36% any-medal on MLE-Bench and top-5% placements in eight live Kaggle competitions.

Problem

Existing ML engineering agents work in isolation and do not use community knowledge (public discussions and shared code) that human Kaggle participants rely on, so they plateau; there is no evaluation framework for community-aware agents.

Approach

Simulate a time-stamped Kaggle community (kernels, discussions, datasets) as an evaluation environment (MLE-Live), and design a multi-agent system (CoMind) whose Coordinator, Analyzer, Idea Proposer, parallel Coding Agents and Evaluator iteratively distill community artifacts into ideas, implement several pipelines in parallel, and publish results back to a shared pool.

Key results

• CoMind achieves a 36.00% Any Medal rate on 75 MLE-Bench competitions, above Neo (34.22%) and R&D-Agent o3 + GPT-4.1 (30.22%).
• On eight live Kaggle competitions, CoMind outperforms 92.6% of human competitors on average, top 5% on three leaderboards and top 1% on one.
• On 20 MLE-Bench-Lite competitions, CoMind achieves a 0.668 average win rate vs. 0.512 for AIDE.

Surprising findings

• Highly upvoted public kernels were often ensembles or reused outputs offering little actionable insight, so vote counts were a poor quality signal.

Limitations

• Evaluator feedback saturates on tiny validation sets, giving unstable signals.
• Underperforms on large-model tasks such as Seq2Seq fine-tuning that exceed the one-hour runtime limit.

Experimental setup

Datasets: MLE-Bench Kaggle public kernels and discussions (MLE-Live)

Benchmarks: MLE-Bench MLE-Bench-Lite CoBench

Metrics: Any Medal win rate above median valid submission rate

Baselines: R&D-Agent ML-Master Neo AIDE OpenHands MLAB AIDE+Code AIDE+RAG

Builds on: MLE-Bench AIDE ReAct

Closest papers by meaning

• AI Research Agents for Machine Learning: Search, Exploration, and Generalization in MLE-bench (NeurIPS 2025 · similarity 0.89)
• MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering (ICLR 2025 · similarity 0.89)
• MLE-STAR: Machine Learning Engineering Agent via Search and Targeted Refinement (NeurIPS 2025 · similarity 0.89)
• MLE-Smith: Scaling MLE Tasks with Automated Multi-agent Pipeline (ICLR 2026 · similarity 0.88)
• MLE-Dojo: Interactive Environments for Empowering LLM Agents in Machine Learning Engineering (NeurIPS 2025 · similarity 0.87)
• MLR-Bench: Evaluating AI Agents on Open-Ended Machine Learning Research (NeurIPS 2025 · similarity 0.87)