Research map / Multi-agent systems and games / LLM agents

RoboWits: Unexpected Challenges for Robotic Creative Problem Solving

NeurIPS 2026 · Multi-agent systems and games › LLM agents · approach: Robust and efficient agent execution · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces RoboWits, a bi-manual simulated manipulation benchmark of 208 creative problem-solving tasks generated by a multi-agent LLM pipeline (from 30 seed tasks), and benchmarks ACT, pi0, pi0.5 and GPT-4o VLM planners.

Problem

Existing robotic benchmarks emphasize skill-level execution under well-specified conditions and do not systematically evaluate cognitive reasoning, creative tool use, or robustness to unexpected challenges in manipulation.

Approach

A multi-agent, foundation-model-driven pipeline (seed generation, verification, mutation via pivot/trap/add strategies, scene generation, metric generation) that scales reasoning-centric bi-manual tasks with graded difficulty and shared evaluation metrics per seed family.

Key results

• pi0 reaches 12.2% average success on seed tasks versus 6.5% on mutations (fine-tuned on 50 demos).
• VLM Planner with oracle states and perfect primitives achieves 60.8% seed SR and 44.4% mutation SR with GPT-4o.
• Closed-loop planning outperforms open-loop by 27.4 points on seed tasks and 29.3 points on mutations.

Surprising findings

• Pre-trained VLAs collapse on mutated tasks, with progress scores frequently halved relative to seed tasks.
• Gemini 3 Pro yields only modest gains over GPT-4o as a VLM planner, suggesting the bottleneck is core reasoning rather than prompting.

Limitations

• Simulation-only evaluation with sim-to-real gap.
• Soft-body simulation is computationally expensive, limiting large-scale evaluation.
• Mutation tasks evaluated with only 5 trials each, and seed tasks with 50 trials on 10 seeds.

Experimental setup

Datasets: RoboWits seed tasks and teleoperated demonstrations

Benchmarks: RoboWits

Metrics: success rate progress score

Baselines: ACT pi0 pi0.5 VLM Planner VLM Controller

Builds on: Genesis RoboTwin 2.0 LIBERO

Closest papers by meaning

• RoboCerebra: A Large-scale Benchmark for Long-horizon Robotic Manipulation Evaluation (NeurIPS 2025 · similarity 0.87)
• RobotSmith: Generative Robotic Tool Design for Acquisition of Complex Manipulation Skills (NeurIPS 2025 · similarity 0.85)
• RoboProcessBench: Benchmarking Process-Aware Understanding in Visual-Language Robotic Manipulation (NeurIPS 2026 · similarity 0.85)
• ClevrSkills: Compositional Language And Visual Reasoning in Robotics (NeurIPS 2024 · similarity 0.85)
• AHA: A Vision-Language-Model for Detecting and Reasoning Over Failures in Robotic Manipulation (ICLR 2025 · similarity 0.85)
• RoboDreamer: Learning Compositional World Models for Robot Imagination (ICML 2024 · similarity 0.85)