Research map / Large language models / Language, safety and interpretability
Leveraging the Dual Capabilities of LLM: LLM-Enhanced Text Mapping Model for Personality Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ETM, a personality detection model that uses Llama3-8B embeddings of concatenated posts with cross-attention fusion over BERT user vectors, plus GPT-4-generated multi-perspective MBTI label interpretations used in contrastive learning with focal loss.
Problem
Personality detection from multiple long social media posts: small encoders like BERT struggle with long multi-document user representations and do not model the relationship between user vectors and MBTI labels well.
Approach
Use an LLM's text embedding capability (Llama3-8B hidden states over concatenated posts, fused via cross-attention with BERT user vectors) and its text generation capability (GPT-4 generated type definitions, tendencies, expression interpretations of MBTI labels) with a contrastive loss that aligns user vectors to label semantics.
Key results
- ETM achieves 77.79 average Macro-F1 on Kaggle and 65.77 on Pandora, versus 72.07 and 63.05 for the prior best TAE.
- Removing the GPT-4 label-interpretation contrastive component drops Kaggle average Macro-F1 by 6.97 points; removing both LLM components drops it by 12.99.
- Llama3 embeddings from the last five layers give the best results; lambda=1 is optimal.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use ETM instead of TAE | personality detection · Kaggle test | Macro-F1 (Avg): 72.07 -> 77.79 | Helped |
| use ETM instead of TAE | personality detection · Pandora test | Macro-F1 (Avg): 63.05 -> 65.77 | Helped |
| add Llama3 user-representation enhancement (ablation) | ETM · Kaggle test | Macro-F1 (Avg): 71.18 -> 77.79 | Helped |
| add GPT-4 label interpretations with contrastive learning (ablation) | ETM · Kaggle test | Macro-F1 (Avg): 72.37 -> 77.79 | Helped |
| include personality-type definitions in label interpretations (ablation) | ETM · Kaggle test | Macro-F1 (Avg): 76.27 -> 77.79 | Helped |
| include thematic tendencies in label interpretations (ablation) | ETM · Kaggle test | Macro-F1 (Avg): 76.52 -> 77.79 | Helped |
| include modes of expression in label interpretations (ablation) | ETM · Kaggle test | Macro-F1 (Avg): 77.03 -> 77.79 | Helped |
| add both Llama3 enhancement and GPT-4 label interpretation/contrastive learning (ablation) | ETM · Kaggle test | Macro-F1 (Avg): 67.68 -> 77.79 | Helped |
| use last 5 rather than last 1 Llama3 embedding layer (ablation) | ETM user representation · Kaggle test | Macro-F1 (Avg): 76.19 -> 77.79 | Helped |
| use last 5 rather than last 2 Llama3 embedding layers (ablation) | ETM user representation · Kaggle test | Macro-F1 (Avg): 76.53 -> 77.79 | Helped |
| use last 5 rather than last 3 Llama3 embedding layers (ablation) | ETM user representation · Kaggle test | Macro-F1 (Avg): 75.41 -> 77.79 | Helped |
| use last 5 rather than last 4 Llama3 embedding layers (ablation) | ETM user representation · Kaggle test | Macro-F1 (Avg): 76.35 -> 77.79 | Helped |
Surprising findings
- Fine-tuned Llama3-8B alone (48.97 avg on Kaggle) performs far below fine-tuned BERT mean pooling (66.04), so LLM-only classification is weak.
Limitations
- Relies on GPT-4 generated label interpretations, which may carry hallucinations despite the authoritative definitions in the prompt.
- Pandora results depend on undersampling to balance classes, which changes the evaluation distribution.
- Llama3 layer selection and lambda are tuned on reported results with limited ablation scope.
Experimental setup
Datasets: Kaggle MBTI Pandora MBTI
Benchmarks: Kaggle MBTI Pandora MBTI
Metrics: Macro-F1
Baselines: SVM XGBoost BiLSTM BERT concat BERT mean AttRCNN AttnSeq Transformer-MD TrigNet D-DGCN TAE Llama3 zero-shot
Builds on: BERT mean pooling contrastive learning (SimCLR) focal loss TAE D-DGCN
Closest papers by meaning
- LLM vs Small Model? Large Language Model Based Text Augmentation Enhanced Personality Detection Model (AAAI 2024 · similarity 0.94)
- Ask, Answer, and Detect: Role-Playing LLMs for Personality Detection with Question-Conditioned Mixture-of-Experts (NeurIPS 2026 · similarity 0.89)
- Data Augmented Graph Neural Networks for Personality Detection (AAAI 2024 · similarity 0.87)
- Knowledge-Enhanced Hierarchical Heterogeneous Graph for Personality Identification with Limited Training Data (AAAI 2025 · similarity 0.85)
- Neuron based Personality Trait Induction in Large Language Models (ICLR 2025 · similarity 0.83)
- Multimodal Fine-Grained Apparent Personality Trait Recognition: Joint Modeling of Big Five and Questionnaire Item-level Scores (AAAI 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.