Research map / Multi-agent systems and games / LLM agents
MARS: Modular Agent with Reflective Search for Automated AI Research
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MARS, an MLE agent combining cost-aware MCTS, modular repository-level code generation, and comparative reflective lesson memory, evaluated on MLE-Bench with Gemini models.
Problem
Autonomous agents struggle with complex machine learning engineering (MLE) tasks because evaluation is computationally expensive, performance attribution is opaque, and monolithic scripts are fragile.
Approach
Combines a budget-aware MCTS whose reward penalizes execution time, a Design-Decompose-Implement modular pipeline with diff-based editing, and comparative reflective memory that distills lessons by diffing the current solution against the best-known one.
Key results
- MARS with Gemini-3-Pro-Preview reaches 56.0% any medal and 31.1% gold rate in controlled 24-hour evaluation on MLE-Bench, versus 37.8% any medal for AIRA-dojo with the same model.
- MARS+ (two concurrent trees, 2x H100) achieves 62.7% any medal rate and 33.8% gold rate, the highest among the reported agents.
- Budget-aware MCTS yields an effective solution rate of 19.5% versus 16.1% for vanilla MCTS.
Surprising findings
- A 4-hour MARS run achieves a higher any-medal rate (28.4%) at lower cost ($9.6) than 24-hour AIRA-dojo baselines, per the paper's cost-controlled evaluation.
- Among 3,611 distilled lessons, 88.34% were judged causally correct by an LLM audit, with 90% agreement on a 20-lesson manual check.
Limitations
- Evaluated only on MLE-Bench, not broader automated AI research tasks.
- High computational and API cost of tree search and comparative distillation.
- Lessons rely on LLM causal attribution, which can hallucinate or misattribute gains.
Experimental setup
Datasets: MLE-Bench MLE-Bench Lite MLE-Bench Medium MLE-Bench High
Benchmarks: MLE-Bench MLE-Bench Lite MLE-Bench Medium MLE-Bench High
Metrics: valid submission rate above median rate any medal rate gold medal rate bronze rate silver rate
Baselines: AIDE AIRA-dojo ML-Master 2.0 ML-Master R&D-Agent InternAgent Famou-Agent Leeroo
Builds on: AIDE AIRA-dojo Reflexion UCT
Closest papers by meaning
- FML-bench: A Controlled Study of AI Research Agent Strategies from the Perspective of Search Dynamics (NeurIPS 2026 · similarity 0.91)
- AI Research Agents for Machine Learning: Search, Exploration, and Generalization in MLE-bench (NeurIPS 2025 · similarity 0.90)
- MLR-Bench: Evaluating AI Agents on Open-Ended Machine Learning Research (NeurIPS 2025 · similarity 0.89)
- ExACT: Teaching AI Agents to Explore with Reflective-MCTS and Exploratory Learning (ICLR 2025 · similarity 0.88)
- From Reproduction to Replication: Evaluating Research Agents with Progressive Code Masking (ICLR 2026 · similarity 0.88)
- ML-Agent: Reinforcing LLM Agents for Autonomous Machine Learning Engineering (ICML 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.