Research map / Vision-language and multimodal / Multimodal LLMs

Unified Personalized Understanding, Generating and Editing

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Interleaved image-text generation with MLLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes OmniPersona, an end-to-end personalization framework for unified LMMs (built on Bagel) that jointly handles personalized understanding, generation, and editing via decoupled concept tokens and an inference-time explicit knowledge replay, plus a new OmniPBench with personalized editing data.

Problem

Unified large multimodal models struggle to model user-specific concepts consistently and controllably across understanding, generation, and editing; prior personalization methods either couple tasks, rely on retrieval, or use opaque learned embeddings, and personalized editing is unexplored.

Approach

Route concept identifier and understanding tokens to an understanding expert and generation tokens to a generation expert (decoupled learnable token subspaces), and at inference use the unified model to parse intent, retrieve learned concept attributes as text, and compose a refined prompt for generation; also add personalized editing data to jointly train all tasks.

Key results

• OmniPersona reaches 0.852 recognition and 0.791 CLIP-I on OmniPBench, the best among unified models reported.
• Explicit knowledge replay raises PARG score from 0.312 to 0.613 (ablation), while Bagel+TP with long text prompts reaches 0.813.
• Personalized editing average score of 0.658 versus 0.558 for GPT-4o+IP, with SEMA-C 0.711.

Surprising findings

• Bagel with text-only concept descriptions (Bagel+TP) outperforms OmniPersona on the PARG score (0.813 vs 0.613) and on several understanding metrics.
• Models trained only on understanding and generation spontaneously acquire some editing ability.

Limitations

• Editing training data is small and mostly removal instructions, so complex backgrounds are hard and the model may alter backgrounds or mislocalize the subject.
• Gains in subject similarity came with reduced instruction-following ability.
• Some baseline numbers come from different settings and reference-only comparisons; editing baselines are not directly compared.

Experimental setup

Datasets: UnifyBench OmniPBench personalized editing data

Benchmarks: OmniPBench

Metrics: Recognition accuracy BLEU GPT score CLIP-I CLIP-T DINO Face similarity (ArcFace) PARG score SEMA-C QUAL-I

Baselines: Yo'LLaVA MC-LLaVA RAP-MLLM Text inversion DreamBooth Chameleon Show-o Bagel Yo'Chameleon Unictoken GPT-4o

Builds on: Bagel Yo'Chameleon UniCTokens UnifyBench DreamBooth

Closest papers by meaning

• MM-R1: Unleashing the Power of Unified Multimodal Large Language Models for Personalized Image Generation (AAAI 2026 · similarity 0.90)
• UniCTokens: Boosting Personalized Understanding and Generation via Unified Concept Tokens (NeurIPS 2025 · similarity 0.89)
• Yo'Chameleon: Personalized Vision and Language Generation (CVPR 2025 · similarity 0.88)
• RAP: Retrieval-Augmented Personalization for Multimodal Large Language Models (CVPR 2025 · similarity 0.87)
• MANZANO: A Simple and Scalable Unified Multimodal Model with a Hybrid Vision Tokenizer (ICLR 2026 · similarity 0.87)
• Ego: Embedding-Guided Personalization of Vision-Language Models (CVPR 2026 · similarity 0.87)