Research map / Vision-language and multimodal / Multimodal LLMs

Face-Human-Bench: A Comprehensive Benchmark of Face and Human Understanding for Multi-modal Assistants

NeurIPS 2025 · Vision-language and multimodal › Multimodal LLMs · approach: Interleaved image-text generation with MLLMs · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces Face-Human-Bench, a 3,600-problem (dev + test, 1,800 each, EN/ZH) multiple-choice benchmark with a three-level taxonomy for face and human understanding, and evaluates 25 MLLMs on it along with CoT/hint effects, position sensitivity, and comparisons to specialist models.

Problem

Existing multimodal LLM benchmarks cover only a narrow set of face and human understanding abilities (e.g., celebrity recognition, action recognition), leaving many face- and human-centric abilities unevaluated and unclear whether MLLMs match specialist models.

Approach

A hierarchical ability taxonomy (2 L1 perspectives, 10 L2, 18 L3 abilities) is used to build multiple-choice problems semi-automatically from 16 public face/human datasets, with image transformations (crop, box-added, identity) and ChatGPT-assisted distractor generation, followed by manual review. The benchmark also introduces a relative position sensitivity score and a relative performance score

Key results

• Overall scores on Face-Human-Bench (EN) range from 27.9% to 76.4%; InternVL-Chat-v1.2-Plus is the top open-source model at 76.4%, while GPT-4o reaches 73.6% face-human overall... (reported overall 70.0% zero-shot for GPT-4o).
• Adding hint plus 1-stage task-specific CoT raises GPT-4o overall from 70.0% to 79.9%, while InternVL-Chat-v1.2-Plus stays flat (75.9-75.0%).
• InternLM-XComposer2-VL-7B has the lowest relative position sensitivity score of 3.7%.

Measured effects reported in the paper

Surprising findings

• Zero-shot GPT-4o does not surpass the best open-source model InternVL-Chat-v1.2-Plus, which the authors attribute to GPT-4o not realizing its potential without CoT.
• CoT and hints hurt or do not help the open-source InternVL-Chat-v1.2-Plus, which often stopped producing rationales.

Limitations

• Multiple-choice format only, which may not reflect open-ended use.
• Specialist comparisons rely on early specialist models and possible data distribution bias between MLLM training data and test datasets.
• Some tasks (e.g., 5 L3 abilities) lack specialist comparison experiments.

Experimental setup

Datasets: CelebA UTKFace RAF-DB FF++ SiW-Mv2 LFW CPLFW CALFW SLLFW MLFW WIDER Attribute HICO-DET

Benchmarks: Face-Human-Bench

Metrics: accuracy weighted accuracy Pearson correlation relative position sensitivity score relative performance score

Baselines: GPT-4o GPT-4V Gemini-1.5-Pro Claude-3.5-Sonnet InternVL-Chat-v1.5 LLaVA-NeXT-34B InternVL-Chat-v1.2-Plus early specialist models

Builds on: MMBench SEED-Bench Multi-modal CoT

Closest papers by meaning

• Human-MME: A Holistic Evaluation Benchmark for Human-Centric Multimodal Large Language Models (ICLR 2026 · similarity 0.87)
• FaceBench: A Multi-View Multi-Level Facial Attribute VQA Dataset for Benchmarking Face Perception MLLMs (CVPR 2025 · similarity 0.87)
• HumanSense: From Multimodal Perception to Empathetic Context-Aware Responses Through Reasoning MLLMs (AAAI 2026 · similarity 0.87)
• SoMi-ToM: Evaluating Multi-Perspective Theory of Mind in Embodied Social Interactions (NeurIPS 2025 · similarity 0.86)
• LifeEval: A Multimodal Benchmark for Assistive AI in Egocentric Daily Life Tasks (CVPR 2026 · similarity 0.86)
• ChatHuman: Chatting about 3D Humans with Tools (CVPR 2025 · similarity 0.86)