AI Paper Atlas

Research map / Multi-agent systems and games / Human-AI and goal-conditioned

CogArena: Benchmarking Multimodal Agents on Interactive Behavioral Experiments

Kianté Fernandez, Caitlin Chen, Jialu Zhou, Bartek Sadowski, Anthony C Miceli, Ian Krajbich

NeurIPS 2026 · Multi-agent systems and games › Human-AI and goal-conditioned · approach: Theory-of-mind social reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces CogArena, a benchmark that tests multimodal agents on interactive behavioral experiments through visual interfaces and time-pressured interaction rather than text prompts.

Problem

Cognitive capabilities of LLMs are usually tested by converting behavioral paradigms into text prompts, which skips the visual interfaces and time-pressured interaction that human participants face; concerns also exist that online behavioral studies can be contaminated by AI agents producing human-l

Approach

Evaluate multimodal agents directly on live, interactive behavioral experiments presented through visual interfaces under time pressure, instead of text-prompt translations of the paradigms.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
judge agents only on task completion rather than also on behavioral signaturesfour economy-tier multimodal models via Browser-Use · CogArena v1Level 1 vs Level 3 scores: 167 of 400 sessions passed all Level 1 checks; only 40% of testable signatures reproduced among themMixed
remove screenshots, giving the agent only DOM-derived page text (ablation)Browser-Use agents · four CogArena taskscomposite score: fell 31--53%Hurt
use gpt-5.6-sol instead of economy-tier panel modelsBrowser-Use agent · CogArena v1 (one session per task)Level 3 score: 0.490 -> 0.656Improved
use gpt-5.6-sol instead of panel models for task completionBrowser-Use agent · CogArena Level 1Level 1 score: 0.750 -> 0.960Improved
change composite weights to (0.10, 0.30, 0.60), (0.20, 0.40, 0.40) or (0, 0, 1) (ablation)CogArena composite score · all-session rankingmodel rankingNo clear effect
change Level 3 grade rule to three alternatives (ablation)CogArena Level 3 · measurable-session rankingagent-floor gap: shifted gap by at most 0.021No clear effect
average over measurable sessions instead of all sessions (ablation)qwen3-vl-235b-instruct · composite scorecomposite score: 26.4 -> 48.3Improved
average over measurable sessions instead of all sessions (ablation)gemini-3-flash-preview · composite scorecomposite score: 58.6 -> 58.7No clear effect
grade capacity signatures vs bias signaturesagents on CogArena · measurable sessionsshare of signatures reproduced: 31% capacity vs 66% biasMixed
compare relative within-session response-time signaturesagents on CogArena · seven RT signaturespass rate: 30 of 217 testable casesHurt

Experimental setup

Benchmarks: CogArena

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy