AI Paper Atlas

Research map / Vision-language and multimodal / Robot action and navigation

ManipEvalAgent: Promptable and Efficient Evaluation Framework for Robotic Manipulation Policies

Yiteng Chen, Huiping Zhuang, WenboLi, Shiyi Wang, Xiangyu Zhao, Qingyao Wu

ICLR 2026 · Vision-language and multimodal › Robot action and navigation · approach: Multimodal prompt-driven manipulation

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
replace fixed-benchmark evaluation with small-batch multi-round ManipEvalAgentpolicy evaluation (ACT) · RoboTwin single-taskevaluation time (min): 167 min -> 42 minImproved
use ManipEvalAgent in multi-task setting instead of full benchmarksRDT evaluation on RoboTwin · multi-taskevaluation time (min): 11102 min -> 97 minImproved
ManipEvalAgent conclusions vs full benchmark success rateRDT, single-task RoboTwin · RoboTwin success ratetrials within error margin: 60% (RDT) to 100% (DP, pi0)Mixed
add retrieval-augmented generation to TaskGen (ablation)TaskGen · task code generationsuccess rate (%): 95% -> 98%Improved
add visual self-check to TaskGen (ablation)TaskGen · task code generationsuccess rate (%): 96% -> 98%Improved
add README.Agent to TaskGen (ablation)TaskGen · task code generationsuccess rate (%): 96% -> 98%Improved
add RAG to ToolGen (ablation)ToolGen · tool code generationsuccess rate (%): 92% -> 96%Improved
add scene perturbations (clutter) to VQA input (ablation)VLM-based VQA tools · RoboTwin 2.0 VQA accuracyaccuracy: 0.997 -> 0.989 (GPT-4o)No clear effect
increase rollouts from 10 to 20/50 for ranking consistency (ablation)ManipEvalAgent · policy ranking Spearman rhoSpearman rho: 0.83 -> 0.79 -> 0.81 (RoboTwin short)No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy