AI Paper Atlas

Research map / Vision-language and multimodal / Multimodal LLMs

Read the Room: Video Social Reasoning with Mental-Physical Causal Chains

Lixing Niu, Jiapeng Li, Xingping Yu, Xinyi Dong, Shu Wang, Ruining Feng, Bo Wu, Ping Wei, Yisen Wang, Lifeng Fan

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces R^3-Bench, a video social reasoning benchmark built around mental-physical causal chains for inferring others' mental states from social cues.

Problem

Existing social reasoning datasets are limited in complexity, scale, and mental-state coverage, falling short of the causal dynamics of real-life interactions, so AI systems struggle to "read the room" from video.

Approach

A video social reasoning benchmark (R^3-Bench) whose items are structured around causal chains linking mental states to physical cues, intended to be richer in complexity, scale, and mental-state coverage than prior datasets.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
SFT on R3-FDTQwen2-VL-7B · R3-Bench-DXoverall accuracy: 74.90 -> 88.74Improved
GRPO (RLFT) on R3-FDTQwen2-VL-7B · R3-Bench-DXoverall accuracy: 74.90 -> 86.88Improved
SFT on R3-FDTQwen2-VL-7B · R3-Bench-Hardaccuracy: 31.96 -> 42.09Improved
GRPO (RLFT) on R3-FDTQwen2-VL-7B · R3-Bench-Hardaccuracy: 31.96 -> 39.87Improved
SFT on R3-FDTQwen2-VL-7B · R3-Bench-DX chain consistencychain consistency: 12.68 -> 35.73Improved
GRPO (RLFT) on R3-FDTQwen2-VL-7B · Social-IQ 2.0accuracy: 61.51 -> 68.40Improved
SFT on R3-FDTQwen2-VL-7B · Social-IQ 2.0accuracy: 61.51 -> 65.38Improved
GRPO (RLFT) on R3-Bench-FDTQwen2-VL-7B · IntentQAaccuracy: 82.38 -> 90.58Improved
SFT on R3-FDTQwen2-VL-7B · IntentQAaccuracy: 82.38 -> 87.77Improved
add subtitles to inputGemini 2.5 Pro · R3-Bench-DXoverall accuracy: 80.79 -> 86.34Improved
add subtitles to inputGPT-4o · R3-Bench-DXoverall accuracy: 82.23 -> 82.64Improved
per-question accuracy vs chain-level consistencyGPT-4o · R3-Bench-DXchain consistency: 82.64 accuracy vs 25.36 consistencyHurt

Experimental setup

Benchmarks: R^3-Bench

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy