Research map / Multi-agent systems and games / LLM agents

MADE: Benchmark Environments for Closed-Loop Materials Discovery

ICML 2026 · Multi-agent systems and games › LLM agents · approach: Software engineering agent benchmarks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces MADE, a modular benchmark framework that simulates closed-loop, oracle-budgeted materials discovery campaigns (MLIP oracle, convex-hull stability) and compares random, generative, planner, MLIP-selector and LLM-agent pipelines.

Problem

Existing computational materials benchmarks evaluate static predictive tasks or isolated sub-tasks, not end-to-end iterative discovery under a limited query budget.

Approach

A gym-like environment where agents propose structures, receive formation-energy oracle feedback, and update a convex hull, scored with discovery-acceleration metrics (AF, EF, AUDC, mSUN) relative to a baseline, with interchangeable planner, generator, filter and selector components plus a ReAct LLM orchestrator.

Key results

• Chemeleon + MLIP selection achieves the highest non-agentic AF of 6.4 and AUDC 0.42 at a 0.1 eV/atom threshold with 50 queries.
• The LLM orchestrator (GPT 5.1) reaches EF 6.0 and AUDC 0.40, comparable to the best modular pipelines, and discovers the most unique space groups (10.4).
• Random generator with LLM planner achieves AF 1.2 and EF 1.5, versus AF 1.0 for the random baseline.

Surprising findings

• MLIP-based ranking degrades at tighter stability thresholds (0.01 eV/atom) due to surrogate error near the convex hull, while LLM planning remains robust.

Limitations

• Generators and MLIPs are trained on Materials Project data, which may simplify discovery relative to real settings.
• Oracle is an MLIP rather than DFT or experiment.
• Only a limited set of strategies is benchmarked; not exhaustive.

Experimental setup

Datasets: Materials Project MP-20

Benchmarks: MADE intermetallic environments (10 ternary, 10 quaternary, 10 quinary systems)

Metrics: acceleration factor enhancement factor AUDC mSUN mean composition L1 unique compositions unique space groups

Baselines: Random generator + random planner Chemeleon + random selection Chemeleon + MLIP selection Random + LLM planner MatterGen OMatG

Builds on: Matbench Discovery ReAct DSPy Chemeleon pymatgen

Closest papers by meaning

• MatExpert: Decomposing Materials Discovery By Mimicking Human Experts (ICLR 2025 · similarity 0.87)
• MANDREL: Modular Reinforcement Learning Pipelines for Material Discovery (AAAI 2024 · similarity 0.87)
• Offline Materials Optimization with CliqueFlowmer (NeurIPS 2026 · similarity 0.87)
• Scalable Diffusion for Materials Generation (ICLR 2024 · similarity 0.86)
• A Sober Look at LLMs for Material Discovery: Are They Actually Good for Bayesian Optimization Over Molecules? (ICML 2024 · similarity 0.85)
• Beyond Drug Discovery: The Nanotechnology Molecular Optimization (NMO) Benchmark (NeurIPS 2026 · similarity 0.85)