Research map / Multi-agent systems and games / LLM agents

MEnvAgent: Scalable Polyglot Environment Construction for Verifiable Software Engineering

ICML 2026 · Multi-agent systems and games › LLM agents · approach: Software engineering agent benchmarks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes MEnvAgent, a multi-agent Planning-Execution-Verification framework with an environment reuse mechanism for scalable polyglot executable environment construction, evaluated on the new 1,000-task MEnvBench and used to build MEnvData-SWE for SFT.

Problem

Constructing executable, verifiable Docker environments for repository-level SWE tasks is labor-intensive and slow, and existing methods are mostly Python-only or lack reliable verification, limiting verifiable training data for SWE agents.

Approach

A multi-agent Planning-Execution-Verification loop with error attribution, plus an environment reuse mechanism that retrieves similar historical environments and adapts them via LLM-generated incremental patches instead of rebuilding from scratch.

Key results

• MEnvAgent improves average F2P from 29.8% (SWE-Factory) to 38.4% (+8.6 points) and PASS from 38.0% to 49.0% on MEnvBench.
• Average time cost drops by 43.0% (6266 s to 3574 s) relative to SWE-Factory.
• Fine-tuning Qwen2.5-Coder-32B-Instruct on MEnvData-SWE trajectories raises SWE-bench Verified resolved rate from 7.5% to 54.6%, matching GPT-4.1 (54.6%).

Surprising findings

• Qwen2.5-Coder-32B-Instruct baseline scores 7.5% on SWE-bench Verified and 0.0% on Multilingual before fine-tuning, indicating base models lack agentic SWE formatting ability.
• Removing the EnvPatchAgent cuts the reuse success rate from 39.0% to 25.0%, and with only 1 instance per repository the reuse rate is negligible.

Limitations

• Results depend on proprietary or API-served LLM backbones (Kimi-K2, Gemini-3-Flash), and reuse benefit grows only with large per-repo data pools.
• Performance degrades for large repositories and C/C++ due to complex build configurations and timeouts.
• Evaluation uses a single benchmark per setting and LLM-based issue quality filtering with DeepSeek-V3.2.

Experimental setup

Datasets: MEnvBench MEnvData-SWE SWE-bench Verified SWE-bench Multilingual

Benchmarks: MEnvBench SWE-bench Verified SWE-bench Multilingual

Metrics: F2P rate PASS rate time cost Reuse Success Rate resolved rate

Baselines: Repo2Run SWE-Bench-Live SWE-Factory w/o EnvPatchAgent w/o Reuse

Builds on: Repo2Run SWE-Bench-Live SWE-Factory OpenHands SWE-bench

Closest papers by meaning

• DOCKSMITH: Scaling Reliable Coding Environments via an Agentic Docker Builder (ICML 2026 · similarity 0.87)
• RepoLaunch: Automating Build and Management of Code Repositories across Languages and Platforms (NeurIPS 2026 · similarity 0.86)
• Nemotron-CORTEXA: Enhancing LLM Agents for Software Engineering Tasks via Improved Localization and Solution Diversity (ICML 2025 · similarity 0.86)
• Structurally Aligned Subtask-Level Memory for Software Engineering Agents (ICML 2026 · similarity 0.86)
• SWE-smith: Scaling Data for Software Engineering Agents (NeurIPS 2025 · similarity 0.86)
• SEC-bench: Automated Benchmarking of LLM Agents on Real-World Software Security Tasks (NeurIPS 2025 · similarity 0.85)