Research map / Vision-language and multimodal / Multimodal LLMs
RePIC: Reinforced Post-Training for Personalizing Multi-Modal Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RePIC, an RL-based (GRPO) post-training framework with verifiable rewards for personalized image captioning in MLLMs, outperforming SFT-based baselines especially in multi-concept settings using only 2K training samples.
Problem
MLLMs struggle to generate faithful personalized image captions, especially with multiple concepts, and SFT-based post-training depends heavily on costly large-scale high-quality personal caption data.
Approach
Replaces SFT with GRPO-based RL using three verifiable rewards: object consistency (yes/no matching of object presence), visual localization (IoU-based bounding box reward), and identity consistency (reward for mentioning all given names, with length regularization), trained on a small 2K-sample dataset.
Key results
- In the 4-concept skip-retrieval setting, RePIC reaches F1 71.0 vs 34.8 for zero-shot Qwen2.5-VL and 21.3 for RAP-Qwen (210K samples).
- In the 2-concept retrieval setting, RePIC achieves F1 95.7 vs 84.5 for RAP-Qwen and 77.5 for zero-shot Qwen2.5-VL.
- On copy-and-paste analysis with wrong single-concept demonstrations, RePIC gets lower BLEU (0.079) than RAP-LLaVA (0.415), RAP-Qwen (0.190) and zero-shot (0.129).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use RL-based RePIC instead of SFT-based RAP-LLaVA | Qwen2.5-VL-Instruct-7B · 4-concept Skip-Retrieval | F1: 34.8 -> 71.0 | Helped |
| Use RL-based RePIC instead of SFT-based RAP-LLaVA | Qwen2.5-VL-Instruct-7B · 2-concept Skip-Retrieval | F1: 96.9 -> 99.4 | Helped |
| Use RL-based RePIC instead of Qwen-2.5 VL | Qwen2.5-VL-Instruct-7B · YoLLaVA | BLEU: 0.210 -> 0.290 | Helped |
| Add the <think> reasoning template (ablation) | RePIC · 2-concept Skip-Retrieval | F1: 99.4 -> 95.2 | Hurt |
| Add the <observe> reasoning template (ablation) | RePIC · 2-concept Skip-Retrieval | F1: 99.4 -> 89.2 | Hurt |
| Add ICT (ablation) | RePIC · 2-concept Skip-Retrieval | F1: 29.2 -> 99.4 | Helped |
| Add OCT (ablation) | RePIC · 2-concept Skip-Retrieval | F1: 84.2 -> 99.4 | Helped |
| Add VLT (ablation) | RePIC · 2-concept Skip-Retrieval | F1: 69.6 -> 99.4 | Helped |
| Use ICT only instead of all reward templates (ablation) | RePIC · 2-concept Skip-Retrieval | F1: 99.4 -> 46.0 | Hurt |
| Use OCT only instead of all reward templates (ablation) | RePIC · 2-concept Skip-Retrieval | F1: 99.4 -> 22.7 | Hurt |
| Use VLT only instead of all reward templates (ablation) | RePIC · 2-concept Skip-Retrieval | F1: 99.4 -> 30.1 | Hurt |
| Add output length regularization (ablation) | RePIC · 2-concept Skip-Retrieval | F1: 95.9 -> 99.4 | Helped |
Surprising findings
- Adding reasoning templates (<think>, <observe>) degraded personal grounding performance.
- Removing the REC/visual localization task from training often caused RL training instability.
- Too high a proportion of identity-consistency (ICT) instructions in training made RL training fail.
Limitations
- Reproduced baselines may not have been exhaustively hyperparameter-tuned.
- Caption faithfulness evaluated only via GPT-4o preference scores due to lack of ground-truth captions.
- Fails when objects are not visible or reference and query views differ greatly (e.g., back vs front view); retrieval noise remains.
Experimental setup
Datasets: COCO Objects365 CelebA Subject200K+ RefCOCO/+/g RAP-MLLM dataset Yo'LLaVA MyVLM DreamBooth InstantFamily
Benchmarks: Yo'LLaVA MyVLM DreamBooth RAP-MLLM dataset (2- and 4-concept)
Metrics: personal grounding precision/recall/F1 BLEU CIDEr METEOR SPICE BERTScore CLIPScore ImageReward Self-BLEU Distinct-n
Baselines: RAP-LLaVA RAP-Qwen PVIT-LLaVA Qwen-2.5 VL zero-shot MyVLM Yo'LLaVA
Builds on: GRPO PPO VLM-R1 Visual-RFT RAP-MLLM
Closest papers by meaning
- CapRL: Stimulating Dense Image Caption Capabilities via Reinforcement Learning (ICLR 2026 · similarity 0.91)
- BalCapRL : A Balanced Framework for RL-Based MLLM Image Captioning (NeurIPS 2026 · similarity 0.90)
- MM-R1: Unleashing the Power of Unified Multimodal Large Language Models for Personalized Image Generation (AAAI 2026 · similarity 0.90)
- Learn from Downstream and Be Yourself in Multimodal Large Language Models Fine-Tuning (ICML 2025 · similarity 0.88)
- Contextualized Visual Personalization in Vision-Language Models (ICML 2026 · similarity 0.88)
- Learning by Correction: Efficient Tuning Task for Zero-Shot Generative Vision-Language Reasoning (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.