Research map / Vision-language and multimodal / Multimodal LLMs

HumanPCR: Probing MLLM Capabilities in Diverse Human-Centric Scenes

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces HumanPCR, a hierarchical benchmark (Perception, Comprehension, Reasoning) for probing MLLMs on human-centric image and video understanding, with 6,000+ multiple-choice questions and 442 open-ended video reasoning questions with human CoT rationales, evaluated on 30+ models.

Problem

Existing benchmarks cover human-centric understanding only narrowly, and video reasoning benchmarks require too little proactive visual evidence, so MLLMs' ability to understand humans in diverse, complex scenes is not well measured.

Approach

A three-level taxonomy (Human-P, Human-C, Human-R) where reasoning items are human-annotated to require integrating multiple visual evidence and proactively extracting evidence beyond what the question cues, with CoT rationales marking key evidence.

Key results

• o4-mini achieves the best Human-P/C macro-average accuracy of 64.13 and 60.42 among listed models, and 58.60% on Human-R per the text, with o3 reaching 59.28% on Human-R in the test-time table.
• InternVL3-78B surpasses o4-mini on Human-P and Human-C among open-source models, with most open-source models below 30% on Human-R.
• Best-of-N test-time scaling yields over 5% gains on three models, while scaling frame count does not significantly improve Human-R reasoning accuracy.

Surprising findings

• Scaling the number of frames does not significantly improve reasoning accuracy on Human-R, and for questions requiring 6+ evidence performance saturates or worsens.
• Text-guided token selection and memory-retrieval context extraction methods underperform on Human-R despite gains on Video-MME, while frame selection and pooling help.

Limitations

• Relies on academic datasets for perception and comprehension levels.
• Uses LLM-based judging (o3-mini) for open-ended reasoning evaluation, lacking fully objective metrics.
• Human-R is limited to a small set of 442 questions and does not cover more professional domains.

Experimental setup

Datasets: academic human-centric datasets YouTube videos

Benchmarks: HumanPCR Human-P HumanPCR Human-C HumanPCR Human-R Video-MME

Metrics: accuracy macro-average accuracy

Baselines: GPT-4o Gemini-2.5-Flash o4-mini o3 Gemini-2.5-Pro Claude-3.7-Sonnet Doubao-1.5-vision-pro Grok-2-Vision Claude-3.5-Sonnet-v2 Gemini-1.5-Pro

Builds on: Chain-of-Thought prompting Video-MME Video-R1

Closest papers by meaning

• Human-MME: A Holistic Evaluation Benchmark for Human-Centric Multimodal Large Language Models (ICLR 2026 · similarity 0.92)
• HumanSense: From Multimodal Perception to Empathetic Context-Aware Responses Through Reasoning MLLMs (AAAI 2026 · similarity 0.90)
• EgoExoBench: A Benchmark for First- and Third-person View Video Understanding in MLLMs (NeurIPS 2025 · similarity 0.89)
• Video-Only ToM: Enhancing Theory of Mind in Multimodal Large Language Models (CVPR 2026 · similarity 0.89)
• SciVideoBench: Benchmarking Scientific Video Reasoning in Large Multimodal Models (ICML 2026 · similarity 0.89)
• LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models (ICLR 2026 · similarity 0.89)