AI Paper Atlas

Research map / Multi-agent systems and games / LLM agents

QiMeng-LibBench: Benchmarking LLM Agents for Library-Scale Cross-Architecture Migration

Weijia Li, KE GAO, Jiajie Li, Han Sun, Yuhe Ding, Yongdong Mai, Yiran Le, Yongjie Qian, Zhibin Zhang, Xinyu Wang, Limin Cheng, Shouxu Kuang and 2 more

ICML 2026 · Multi-agent systems and games › LLM agents · approach: Robust and efficient agent execution

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces QiMeng-LibBench, a benchmark for evaluating LLM agents on library-scale cross-architecture migration of high-performance code with ISA-specific SIMD intrinsics.

Problem

Cross-architecture migration of high-performance libraries requires handling library-scale dependencies and performance-critical SIMD kernels, but existing LLM agent evaluations are limited to function-level tasks or scalar code.

Approach

A benchmark targeting library-scale migration including ISA-specific SIMD intrinsics, assessing agents on both migration correctness/speed and peak performance rather than function-level scalar code.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use GPT-5 as the underlying LLM instead of the second-best modelagents · QM-LibBenchSR@1: average +10.39 SR@1Improved
use open-weights Qwen3-coder-480b instead of proprietary modelsOpenHands · QM-LibBenchSR@1: 17.65 overall SR@1Hurt
pair Gemini-3-pro-preview with the weaker SWE-Agent instead of OpenHandsGemini-3-pro-preview · QM-LibBenchoverall SR@1: 36.47 -> 3.53Hurt
migrate to emerging ISAs (RISC-V, LoongArch) instead of x86->ARMLLM agents · QM-LibBenchaverage accuracy: 31.37% -> 18.14% (RISC-V), 20.59% (LoongArch)Hurt
migrate OpenCV instead of ncnnLLM agents · QM-LibBenchaverage SR: 23.37% (ncnn) vs 14.68% (OpenCV)Hurt
raise speedup threshold p from 0 to 1.0all agent-LLM combinations · QM-LibBenchfastpHurt
use OpenHands instead of JoyCode with GPT-5OpenHands · QM-LibBenchcost on successful migrations: up to 3x longer, 4x more API callsHurt
use OpenCode instead of other agents with GPT-5OpenCode · QM-LibBench OpenCVSR@1 and cost: 33.33 SR@1, time 429, money 1.75Improved
failed trajectories vs successful onesSWE-Agent, JoyCode, OpenHands · QM-LibBenchmoney cost (SR@1): OpenHands 2.04 -> 2.19Hurt

Experimental setup

Benchmarks: QiMeng-LibBench

Metrics: migration correctness migration speed peak performance

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy