Research map / Vision-language and multimodal / Multimodal LLMs

PersonaVLM: Long-Term Personalized Multimodal LLMs

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Unified any-modality language models · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes PersonaVLM, a Qwen2.5-VL-7B-based agent with multi-type memory, retrieval-driven multi-step reasoning, and an EMA-based Big Five personality tracker, trained with SFT then GRPO, plus a new long-term personalization benchmark Persona-MME.

Problem

MLLMs struggle to personalize responses over long-term interactions because existing methods are static, single-turn, and fail to track evolving preferences and personality over time.

Approach

A unified agent framework that proactively writes multimodal memories (core, semantic, episodic, procedural), performs agentic time-aware retrieval in multiple reasoning steps, and updates a momentum-smoothed Big Five personality profile that conditions responses.

Key results

• Under 128k context, PersonaVLM (RL) reaches 71.05 overall on Persona-MME vs 59.01 for the RAG Qwen2.5-VL-7B baseline (+12.04) and 47.28 on PERSONAMEM vs GPT-4o's 45.32.
• On personalized alignment, PersonaVLM scores 92.22 on Persona-MME 128k and 49.60 overall on P-SOUPS, leading Qwen3-30B-A3B (47.14).
• In pairwise open-ended evaluation judged by Gemini-2.5-Pro, PersonaVLM wins 79% against GPT-4o with a 16% loss rate.

Surprising findings

• Naive RAG degrades Qwen2.5-VL-7B on preference tasks in short-context settings by up to 9.33%.
• Language-centric Qwen3-30B-A3B outperforms the larger multimodal InternVL3-38B on personality alignment.

Limitations

• Relies on an LLM judge (Qwen3-30B-A3B) for accuracy and consistency rewards during RL.
• Persona-MME and the training data are synthetic, generated by commercial and proprietary models.

Experimental setup

Datasets: PersonaHub PERSONAMEM P-SOUPS

Benchmarks: Persona-MME PERSONAMEM P-SOUPS

Metrics: accuracy win rate

Baselines: GPT-4o Qwen2.5-VL-7B InternVL3-8B InternVL3-38B OneVision-1.5-8B Qwen3-30B-A3B Self-Critic Few-Shot

Builds on: GRPO MemGPT Qwen2.5-VL PersonaHub PERSONAMEM

Closest papers by meaning

• PersonalLLM: Tailoring LLMs to Individual Preferences (ICLR 2025 · similarity 0.91)
• MMPB: It’s Time for Multi-Modal Personalization (NeurIPS 2025 · similarity 0.89)
• Mem-PAL: Towards Memory-based Personalized Dialogue Assistants for Long-term User-Agent Interaction (AAAI 2026 · similarity 0.89)
• PICLe: Eliciting Diverse Behaviors from Large Language Models with Persona In-Context Learning (ICML 2024 · similarity 0.89)
• Do LLMs Recognize Your Preferences? Evaluating Personalized Preference Following in LLMs (ICLR 2025 · similarity 0.89)
• Personalize-then-Store: Benchmarking and Learning Personalized Memory for Long-Horizon Agents (NeurIPS 2026 · similarity 0.88)