Research map / Multi-agent systems and games / LLM agents

IS-Bench: Evaluating Interactive Safety of VLM-Driven Embodied Agents in Daily Household Tasks

AAAI 2026 · Multi-agent systems and games › LLM agents · approach: Software engineering agent benchmarks · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes IS-Bench, a multimodal interactive simulator benchmark (161 OmniGibson household scenarios, 388 risks) that scores VLM embodied agents on whether safety goals are met before/after risk-prone steps, finding low safe success rates.

Problem

Existing embodied safety benchmarks are static, text-only or single-image, and termination-oriented, so they miss dynamic risks emerging from agent actions and temporary unsafe states during household task execution.

Approach

Process-oriented evaluation in a physics simulator: each safety goal is bound to a trigger (pre- or post-caution relative to a risk-prone action) and checked during execution, combined with multi-view visual inputs and three levels of safety reminders (implicit, CoT, explicit).

Key results

• GPT-4o reaches 81.3% SR but only 33.8% SSR under the implicit reminder (L1), showing many tasks are completed by violating safety constraints.
• Safety CoT reminder raises SRec (All) by about 9.3% on average but lowers SR by about 9.4% on average; GPT-4o SR falls from 81.3% to 53.8%.
• With explicit safety goals (L3), GPT-4o and Gemini-2.5-pro reach SRec (All) of 91.2% and 92.2%, while their safety awareness scores are low, pointing to risk perception as the bottleneck.

Surprising findings

• Safety-aware CoT improves safety recall but substantially reduces task success rate, especially for GPT-4o.
• Providing ground-truth initial scene setup inflates safe success, suggesting data leakage that bypasses genuine risk perception.

Limitations

• Simulation does not model human user activities that can create dynamic risks.
• Gap between simulation and real-world environments remains.
• Evaluation focuses on benchmarking; no method is proposed to improve interactive safety.

Experimental setup

Datasets: BEHAVIOR-1K

Benchmarks: IS-Bench

Metrics: Success Rate (SR) Safe Success Rate (SSR) Safety Recall (SRec) Safety Awareness (SA)

Baselines: GPT-4o Gemini-2.5-pro Gemini-2.5-flash Claude-3.7-Sonnet Qwen2.5-VL-72B-Ins InternVL3-78B Llama-3.2-90B-Vision-Ins GPT-4o-mini

Builds on: OmniGibson BEHAVIOR-1K PDDL

Closest papers by meaning

• AGENTSAFE: Benchmarking the Safety of Embodied Agents on Hazardous Instructions (CVPR 2026 · similarity 0.93)
• SENTINEL: A Multi-Level Formal Framework for Safety Evaluation of Foundation Model-based Embodied Agents (NeurIPS 2026 · similarity 0.90)
• HAZARD Challenge: Embodied Decision Making in Dynamically Changing Environments (ICLR 2024 · similarity 0.88)
• OpenAgentSafety: A Comprehensive Framework For Evaluating Real-World AI Agent Safety (ICLR 2026 · similarity 0.88)
• RiOSWorld: Benchmarking the Risk of Multimodal Computer-Use Agents (NeurIPS 2025 · similarity 0.87)
• Building a Foundational Guardrail for General Agentic Systems via Synthetic Data (ICLR 2026 · similarity 0.87)