Research map / Multi-agent systems and games / Human-AI and goal-conditioned
CogArena: Benchmarking Multimodal Agents on Interactive Behavioral Experiments
Kianté Fernandez, Caitlin Chen, Jialu Zhou, Bartek Sadowski, Anthony C Miceli, Ian Krajbich
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces CogArena, a benchmark that tests multimodal agents on interactive behavioral experiments through visual interfaces and time-pressured interaction rather than text prompts.
Problem
Cognitive capabilities of LLMs are usually tested by converting behavioral paradigms into text prompts, which skips the visual interfaces and time-pressured interaction that human participants face; concerns also exist that online behavioral studies can be contaminated by AI agents producing human-l
Approach
Evaluate multimodal agents directly on live, interactive behavioral experiments presented through visual interfaces under time pressure, instead of text-prompt translations of the paradigms.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| judge agents only on task completion rather than also on behavioral signatures | four economy-tier multimodal models via Browser-Use · CogArena v1 | Level 1 vs Level 3 scores: 167 of 400 sessions passed all Level 1 checks; only 40% of testable signatures reproduced among them | Mixed |
| remove screenshots, giving the agent only DOM-derived page text (ablation) | Browser-Use agents · four CogArena tasks | composite score: fell 31--53% | Hurt |
| use gpt-5.6-sol instead of economy-tier panel models | Browser-Use agent · CogArena v1 (one session per task) | Level 3 score: 0.490 -> 0.656 | Improved |
| use gpt-5.6-sol instead of panel models for task completion | Browser-Use agent · CogArena Level 1 | Level 1 score: 0.750 -> 0.960 | Improved |
| change composite weights to (0.10, 0.30, 0.60), (0.20, 0.40, 0.40) or (0, 0, 1) (ablation) | CogArena composite score · all-session ranking | model ranking | No clear effect |
| change Level 3 grade rule to three alternatives (ablation) | CogArena Level 3 · measurable-session ranking | agent-floor gap: shifted gap by at most 0.021 | No clear effect |
| average over measurable sessions instead of all sessions (ablation) | qwen3-vl-235b-instruct · composite score | composite score: 26.4 -> 48.3 | Improved |
| average over measurable sessions instead of all sessions (ablation) | gemini-3-flash-preview · composite score | composite score: 58.6 -> 58.7 | No clear effect |
| grade capacity signatures vs bias signatures | agents on CogArena · measurable sessions | share of signatures reproduced: 31% capacity vs 66% bias | Mixed |
| compare relative within-session response-time signatures | agents on CogArena · seven RT signatures | pass rate: 30 of 217 testable cases | Hurt |
Experimental setup
Benchmarks: CogArena
Closest papers by meaning
- WorldMemArena: Evaluating Multimodal Agent Memory Through Action–World Interaction (NeurIPS 2026 · similarity 0.89)
- MiroEval: Benchmarking Multimodal Deep Research Agents in Process and Outcome (NeurIPS 2026 · similarity 0.89)
- Sketchtopia: A Dataset and Foundational Agents for Benchmarking Asynchronous Multimodal Communication with Iconic Feedback (CVPR 2025 · similarity 0.88)
- BEAR: Dissecting Embodied Abilities in Multimodal Language Models through Skill-level Evaluation and Diagnosis (ICML 2026 · similarity 0.87)
- OdysSim: Building Foundation Models for Human Behavior Simulation (NeurIPS 2026 · similarity 0.87)
- HumanSense: From Multimodal Perception to Empathetic Context-Aware Responses Through Reasoning MLLMs (AAAI 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
