AI Paper Atlas

Research map / Vision-language and multimodal / Human and embodied modeling

Aligning MLLMs with the Latent Structure of Human Cognition via Behavior-Derived Semantic Dimensions

Ning E, Changde Du, Yizhuo Lu, Huiguang He

NeurIPS 2026 · Vision-language and multimodal › Human and embodied modeling · approach: Language-grounded visual concept learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes aligning MLLMs with the latent multidimensional structure underlying human similarity judgements by building on behavior-derived semantic dimensions.

Problem

Unclear whether MLLMs' internal representations and decision criteria align with the latent structure of human cognition, particularly the multidimensional mental representations underlying similarity judgements.

Approach

A framework that aligns MLLM representations with behavior-derived semantic dimensions recovered from human similarity judgements, building on a previous (unspecified) line of work on such dimensions.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
fine-tune on dimension-grounded rationales (CogAlign) instead of CoT promptingQwen3-VL-4B · O1O human consistencyaccuracy (%): 53.37 -> 56.81Improved
CogAlign versus Direct prompt baselineQwen3-VL-30B-A3B · O1O human consistencyaccuracy (%): 37.09 -> 56.21Improved
fine-tune with CogAlign versus CoT baseline as model size growsQwen3-VL (4B, 8B, 30B-A3B) · O1O human consistencyaccuracy (%): 53.37 -> 56.81; 52.28 -> 56.99; 55.38 -> 56.21Improved
add CogAlign fine-tuningQwen3-VL · MMMU valaccuracy (%): 53.22 -> 57.56 (30B-A3B)Improved
add CogAlign fine-tuningQwen3-VL · MMBench en testaccuracy (%): 87.44 -> 83.86 (30B-A3B); 82.96 -> 81.39 (4B)Hurt
constrain inference-time CoT to the top-1 salient dimension d* (ablation)Qwen3-VL-4B · O1O human consistencyaccuracy (%): 53.37 -> 49.01Hurt
prompt with full 66-dimension space at inference (FS-CoT) (ablation)Qwen3-VL-4B · O1O human consistencyaccuracy (%): 53.37 -> 52.70No clear effect
apply CogAlign to other model families (ablation)Qwen2.5-VL-7B and Gemma3-12B · O1O validation subsetaccuracy (%): 56.63 -> 59.29 (Qwen2.5-VL); 55.77 -> 57.28 (Gemma3)Improved
use rationales generated by DeepSeek-V4-Flash instead of Qwen2.5-72B (ablation)CogAlign Qwen3-VL-4B · O1O human consistencyaccuracy (%): 56.81 -> 56.52No clear effect
align Qwen2.5-VL-7B with CogAlign for behavioral geometryQwen2.5-VL-7B · 48-object choice-based RSM correlationcorrelation (%): 73.30 -> 80.93Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy