Research map / Large language models / Retrieval and knowledge

Ask, Answer, and Detect: Role-Playing LLMs for Personality Detection with Question-Conditioned Mixture-of-Experts

NeurIPS 2026 · Large language models › Retrieval and knowledge · approach: Explanatory and multiple-choice QA benchmarks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ROME, which uses GPT-4o role-play to answer MBTI questionnaire items from user posts offline, then trains a question-conditioned mixture-of-experts with BERT to predict those answers and fuses them with user representations for MBTI personality detection.

Problem

Personality detection from social media posts is limited by label scarcity and under-specified mappings from user language to psychological constructs, as existing "posts → user vector → labels" models rely only on coarse MBTI labels.

Approach

Use an LLM to role-play users and simulate psychometric questionnaire answers as intermediate supervision (offline, training only), then train a question-conditioned MoE to predict these answers from posts and fuse the reliability- and importance-weighted answer vector with the user representation under multi-task learning.

Key results

• ROME reaches 89.78% average Macro-F1 on Kaggle versus 77.79% for ETM, a reported 15.41% relative improvement.
• On Pandora, ROME reaches 69.04% average Macro-F1 versus 65.77% for ETM (4.97% relative gain), but is weaker on I/E (49.05 vs 68.57 for ETM).
• Ablation on Kaggle: removing the pretraining stage drops average Macro-F1 to 69.21%; using only questionnaire evidence gives 75.12%.

Surprising findings

• On Pandora, ROME is far below ETM on the I/E dimension (49.05 vs 68.57 F1), and it underperforms on P/J on Kaggle as well.
• Using only post representations without questionnaire evidence drops Kaggle average to 65.77%, lower than the full model's gain would suggest from the evidence alone.

Limitations

• Performance on Pandora is weaker than ETM on I/E and P/J, so gains are not uniform across datasets and dimensions.
• Role-play answers from GPT-4o may be noisy and their quality depends on the LLM used, as shown by the GPT-3.5 vs GPT-4o comparison.

Experimental setup

Datasets: Kaggle MBTI (PersonalityCafe) Pandora Reddit MBTI

Benchmarks: Kaggle MBTI Pandora

Metrics: Macro-F1 per MBTI dimension average Macro-F1

Baselines: XGBoost BiLSTM BERTconcat BERTmean AttRCNN AttnSeq Transformer-MD TrigNet D-DGCN TAE ETM ChatGPT-3.5

Builds on: TAE ETM multi-gate mixture-of-experts BERT

Closest papers by meaning

• Leveraging the Dual Capabilities of LLM: LLM-Enhanced Text Mapping Model for Personality Detection (AAAI 2025 · similarity 0.89)
• LLM vs Small Model? Large Language Model Based Text Augmentation Enhanced Personality Detection Model (AAAI 2024 · similarity 0.89)
• Data Augmented Graph Neural Networks for Personality Detection (AAAI 2024 · similarity 0.87)
• Multimodal Fine-Grained Apparent Personality Trait Recognition: Joint Modeling of Big Five and Questionnaire Item-level Scores (AAAI 2025 · similarity 0.84)
• PersonalLLM: Tailoring LLMs to Individual Preferences (ICLR 2025 · similarity 0.84)
• PICLe: Eliciting Diverse Behaviors from Large Language Models with Persona In-Context Learning (ICML 2024 · similarity 0.84)