Research map / Vision-language and multimodal / Human and embodied modeling
Human-centered Interactive Learning via MLLMs for Text-to-Image Person Re-identification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ICL, which uses a LoRA-fine-tuned MLLM (Qwen2-VL-7B) for test-time multi-round human-centered VQA to refine text queries for text-to-image person re-ID, plus Reorganization Data Augmentation of training texts.
Problem
Text-to-image person re-identification struggles to distinguish challenging candidates because offline cross-modal models and low-quality, vague training texts limit query-image alignment.
Approach
A plug-and-play test-time module asks an MLLM yes/no anchor-localization questions and human-centered VQA over top-ranked candidates, merges answers into an enriched query for re-ranking, and an MLLM-based decompose-rewrite-reorder augmentation diversifies training captions.
Key results
- With THI, ICL⋆ reaches Rank-1 of 77.91% on CUHK-PEDES, 69.02% on ICFG-PEDES and 70.55% on RSTPReid in group ❶ (vs. RDE 75.94/67.68/65.35).
- On UFine6926, ICL⋆ with ReID pre-training reaches Rank-1 91.78% and mAP 90.33%, versus RDE 87.60 Rank-1 and 86.10 mAP.
- THI is plug-and-play: it raises average Rank-1/mAP gains of +2.57 (CLIP), +1.86 (IRRA), +1.41 (RDE) points across three coarse benchmarks.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add LoRA fine-tuning (ablation) | ICL with RDA and THI · CUHK-PEDES | Rank-1: 76.38 -> 77.91 | Helped |
| add LoRA fine-tuning (ablation) | ICL with RDA and THI · ICFG-PEDES | Rank-1: 67.92 -> 69.02 | Helped |
| add LoRA fine-tuning (ablation) | ICL with RDA and THI · RSTPReid | Rank-1: 69.00 -> 70.55 | Helped |
| add THI (ablation) | ICL with RDA, without LoRA fine-tuning · CUHK-PEDES | Rank-1: 76.41 -> 76.38 | No clear effect |
| add THI (ablation) | ICL with RDA, without LoRA fine-tuning · ICFG-PEDES | Rank-1: 68.11 -> 67.92 | No clear effect |
| add THI (ablation) | ICL with RDA, without LoRA fine-tuning · RSTPReid | Rank-1: 67.70 -> 69.00 | Helped |
| add RDA (ablation) | ICL without THI or LoRA fine-tuning · CUHK-PEDES | Rank-1: 75.94 -> 76.41 | Helped |
| add RDA (ablation) | ICL without THI or LoRA fine-tuning · ICFG-PEDES | Rank-1: 67.68 -> 68.11 | Helped |
| add RDA (ablation) | ICL without THI or LoRA fine-tuning · RSTPReid | Rank-1: 65.35 -> 67.70 | Helped |
Surprising findings
- THI slightly degraded Rank-5 and Rank-10 on some settings, which the authors attribute to error accumulation when the anchor image is wrongly located with only 5 rounds.
Limitations
- Errors in anchor localization accumulate across interaction rounds, degrading Rank-5/10.
- Relies on MLLM inference at test time, adding inference cost despite efficiency gating.
- Original Han et al. method leaks ground truth; the proposed method's reliance on an MLLM answerer's reliability to domain-shifted ReID images is limited without SFT.
Experimental setup
Datasets: CUHK-PEDES ICFG-PEDES RSTPReid UFine6926
Benchmarks: CUHK-PEDES ICFG-PEDES RSTPReid UFine6926
Metrics: Rank-1 Rank-5 Rank-10 mAP mINP
Baselines: IRRA RDE TBPS RaSa CFAM NAM APTM LCR2S IVT LGUR SSAN
Builds on: RDE CLIP LoRA IRRA
Closest papers by meaning
- LLaVA-ReID: Selective Multi-image Questioner for Interactive Person Re-Identification (ICML 2025 · similarity 0.90)
- Noisy-Correspondence Learning for Text-to-Image Person Re-identification (CVPR 2024 · similarity 0.89)
- Modeling Thousands of Human Annotators for Generalizable Text-to-Image Person Re-identification (CVPR 2025 · similarity 0.88)
- Chat-based Person Retrieval via Dialogue-Refined Cross-Modal Alignment (CVPR 2025 · similarity 0.87)
- ReID5o: Achieving Omni Multi-modal Person Re-identification in a Single Model (NeurIPS 2025 · similarity 0.87)
- Multi-Prompts Learning with Cross-Modal Alignment for Attribute-Based Person Re-identification (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.