Research map / Vision-language and multimodal / Multimodal LLMs

PersonaX: Multimodal Datasets with LLM-Inferred Behavior Traits

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Interleaved image-text generation with MLLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Releases PersonaX, two multimodal datasets (CelebPersona with 9444 celebrities and AthlePersona with 4181 athletes) pairing LLM-inferred Big Five trait scores and descriptions with facial embeddings and biographical data, plus a two-level analysis (independence tests and a multimodal multi-measurement causal representation learning method with identifiability theory).

Problem

Existing resources rarely combine behavioral trait descriptors with complementary modalities such as facial attributes and biographical information, limiting cross-modal and causal analysis of human behavior traits.

Approach

Builds multimodal persona datasets whose Big Five trait scores and text descriptions are inferred by three LLMs, then analyzes them with statistical independence tests and a new multi-modality multi-measurement causal representation learning framework with identifiability guarantees.

Key results

• On synthetic variant MNIST, the proposed CRL method reaches R^2 0.96 and MCC 0.92, versus MMCRL at R^2 0.90 and MCC 0.85.
• In LLM selection on 100 sampled individuals per dataset, ChatGPT-4o and Gemini-2.5-Pro achieved the top overall scores (0.96 on CelebPersona), and ChatGPT-4o scored 0.93 on AthlePersona.
• Independence tests on the real data find gender and occupation strongly dependent on nearly all trait scores in CelebPersona, while league and birth year matter more for AthlePersona.

Surprising findings

• Gemini-2.5-Pro, a top-ranked Arena model, had a much longer generation time (about 23 s on CelebPersona) than most other candidates.

Limitations

• AthlePersona covers only male athletes and CelebPersona only high-visibility celebrities, so results are population-specific.
• Traits are inferred by LLMs from static public data without longitudinal validation.
• Four additional leagues were excluded due to consent requirements.

Experimental setup

Datasets: CelebA Wikidata AthlePersona CelebPersona Colored MNIST Fashion MNIST MNIST

Benchmarks: Colored MNIST + Fashion MNIST synthetic causal benchmark AthlePersona CelebPersona

Metrics: R^2 MCC generation time missing rate indecisive rate privacy preservation output formatting context consistency factual accuracy overall score

Baselines: BetaVAE MCL MMCRL

Builds on: CelebA MMCRL Multimodal Contrastive Learning BetaVAE ImageBind

Closest papers by meaning

• Pose as a Modality: A Psychology-Inspired Network for Personality Recognition with a New Multimodal Dataset (AAAI 2025 · similarity 0.88)
• Multimodal Fine-Grained Apparent Personality Trait Recognition: Joint Modeling of Big Five and Questionnaire Item-level Scores (AAAI 2025 · similarity 0.87)
• PICLe: Eliciting Diverse Behaviors from Large Language Models with Persona In-Context Learning (ICML 2024 · similarity 0.86)
• Personality Alignment of Large Language Models (ICLR 2025 · similarity 0.85)
• LLM Generated Persona is a Promise with a Catch (NeurIPS 2025 · similarity 0.85)
• Persona Vectors: Monitoring and Controlling Character Traits in Language Models (NeurIPS 2026 · similarity 0.85)